0

I have created a DAG in Airflow that will detect a file in Azure blob storage. This is a sensor that will detect the existence of a blob in Azure blob storage.

But it is failing repeatedly with the below error:

ValueError: client_id should be the id of a Microsoft Entra application

I have installed the required packages for it. apache-airflow-providers-microsoft-azure==12.7.0

I have created a connection to the Microsoft Azure blob using the below headers:

{
"account_name": "source",
"tenant_id": "\<tenant_id of the service principal\>",
"client_id": "\<client_id of the service principal\>",
"client_secret": "***"
}

Yet, it is throwing an error with the below code:

from airflow.sdk import dag, task
from airflow.providers.microsoft.azure.sensors.wasb import WasbBlobSensor
from datetime import datetime, timedelta

default_args = {
    "owner": "airflow",
    "retries": 6,
    "retry_delay": timedelta(seconds=30)
}

@dag(dag_id="azure_blob_sensor_dag",default_args=default_args,start_date=datetime(2025, 10, 13),schedule=None,catchup=False,tags=["azure", "sensor"])
def sensor_dag():
    wait_for_blob = WasbBlobSensor(
        wasb_conn_id="azure_blob_connection",
        container_name="source",
        blob_name="citizens/citizens.csv",
        poke_interval=30,
        timeout=5 * 60
    )

    @task
    def task_action():
        print("File is present in ADLS")

    wait_for_blob >> task_action()

sensor_dag()

I am not able to understand why it is throwing an error:

ValueError: client_id should be the id of a Microsoft Entra application

Because I have already mentioned the client_id of the service principal.

What else is it expecting?

How can I create my first ADLS sensor?

Improve this question
5
  • What is "service principal"? Do you mean "service principle"? Commented Oct 13 at 5:26
  • Where are you getting your client_id from? Maybe take a look at this. spletzer.com/2025/01/… Commented Oct 13 at 5:53
  • Try creating the connection in following way: Using Service Principal Authentication set up your connection as follows: Connection Type: wasb Login: <client_id of the service principal> Password: <client_secret of the service principal> Host: <storage_account_name> or <storage_account_name>.blob.core.windows.net Extra: { "tenant_id": "<tenant_id of the service principal>" } Commented Oct 13 at 6:23
  • Hi @PeterMortensen: It is service principal as per azure documentation. learn.microsoft.com/en-us/azure/devops/integrate/get-started/… Commented Oct 13 at 16:32
  • Hi @AbdullahNafees: Yes, I followed the same steps as per the your suggested document. I got this client_id from microsoft entra id only. Commented Oct 13 at 16:33
Related questions
1
Airflow Task triggered manually but remains in queued state
4
can we parameterize the airflow schedule_interval dynamically reading from the variables instead of passing as the cron expression
0
Airflow Hash "#" in day-of-week field not running appropriately
Load 6 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.