{"id":762,"library":"google-cloud-dataflow-client","title":"Google Cloud Dataflow Client","description":"The `google-cloud-dataflow-client` is the Python client library for interacting with the Google Cloud Dataflow API. It allows users to programmatically manage Dataflow jobs, such as listing existing jobs, getting job details, or submitting new ones based on templates. This library provides an interface to the Dataflow service API, distinct from the `apache-beam` SDK, which is used for defining Dataflow pipelines themselves. It is part of the broader `google-cloud-python` ecosystem and generally follows its release cadence, receiving regular updates for bug fixes and new features.","status":"active","version":"0.11.0","language":"python","source_language":"en","source_url":"https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-dataflow-client","tags":["google-cloud","dataflow","client","gcp","job-management"],"install":[{"cmd":"pip install google-cloud-dataflow-client","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Core library for Google API clients, providing common functionality like transport, authentication, and retry logic.","package":"google-api-core"}],"imports":[{"wrong":"from google.cloud.dataflow_v1beta3 import JobsClient","symbol":"JobsClient","correct":"from google.cloud.dataflow_v1beta3 import JobsClient"}],"quickstart":{"code":"import os\nfrom google.cloud.dataflow_v1beta3 import JobsClient\nfrom google.cloud.dataflow_v1beta3.types import ListJobsRequest\n\n# Set your Google Cloud Project ID (e.g., via GOOGLE_CLOUD_PROJECT environment variable)\nproject_id = os.environ.get(\"GOOGLE_CLOUD_PROJECT\", \"your-gcp-project-id\")\nregion = \"us-central1\" # Dataflow jobs are regional, specify the region where your jobs run\n\nif project_id == \"your-gcp-project-id\":\n    print(\"WARNING: Please set the GOOGLE_CLOUD_PROJECT environment variable or replace 'your-gcp-project-id'.\")\n    exit()\n\ntry:\n    # Initialize the client (will use Application Default Credentials by default)\n    client = JobsClient()\n\n    # Create a request to list jobs in a specific project and region\n    request = ListJobsRequest(project_id=project_id, location=region)\n\n    print(f\"Listing Dataflow jobs for project '{project_id}' in region '{region}':\")\n    response = client.list_jobs(request=request)\n\n    jobs_found = False\n    for job in response.jobs:\n        print(f\"  Job ID: {job.id}, Name: {job.name}, Type: {job.type}, State: {job.current_state}\")\n        jobs_found = True\n\n    if not jobs_found:\n        print(\"  No Dataflow jobs found.\")\n\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")\n    print(\"Ensure you have authenticated with `gcloud auth application-default login` and enabled the Dataflow API in your project.\")","lang":"python","description":"This quickstart demonstrates how to initialize the Dataflow Jobs client and list existing Dataflow jobs in a specified Google Cloud project and region. Ensure your `GOOGLE_CLOUD_PROJECT` environment variable is set and you have authenticated via `gcloud auth application-default login`."},"warnings":[{"fix":"Use `apache-beam` for writing and defining your Dataflow pipelines. Use `google-cloud-dataflow-client` for programmatic control over the Dataflow service itself (e.g., listing, launching, or updating jobs).","message":"This library (`google-cloud-dataflow-client`) is for *managing* Dataflow jobs and interacting with the Dataflow service. It is not for *defining* data processing pipelines. Pipeline definition is done using the Apache Beam SDK (`apache-beam` library).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always refer to the official documentation or the library's source code for the exact and recommended import paths for the specific API version you intend to use. For `google-cloud-dataflow-client` version `0.11.0`, the primary API surface is `dataflow_v1beta3`.","message":"Google Cloud client libraries often expose API versions (e.g., `v1beta3`) directly in their import paths. Using an incorrect or deprecated API version in your import statement (e.g., `from google.cloud.dataflow_v1` instead of `dataflow_v1beta3`) can lead to `ImportError` or unexpected API behavior.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure that the `location` parameter is explicitly provided in your client calls (e.g., `client.list_jobs(project_id=project_id, location='us-central1')`) and matches the region where your Dataflow jobs are deployed or expected to run.","message":"Dataflow operations are regional. When interacting with the Dataflow service, it is crucial to specify the correct `location` (region) for listing or creating jobs. Failing to do so may result in not finding expected jobs or encountering errors.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Monitor official Google Cloud Dataflow documentation and release notes for updates to the API and client library. Be prepared to adapt your code if `v1beta3` features are modified or a new, more stable API version becomes available and is recommended.","message":"The Dataflow API `v1beta3` is a beta release and, as such, its methods and types are subject to backward-incompatible changes without a long deprecation period. While generally stable, rely on `beta` components with caution in production environments.","severity":"deprecated","affected_versions":"All versions of `google-cloud-dataflow-client` that expose `v1beta3` clients (e.g., 0.11.0)"}],"env_vars":null,"search_vec":"'allow':25 'apach':61 'apache-beam':60 'api':23,56 'base':44 'beam':62 'broader':77 'bug':93 'cadenc':88 'client':4,10,14,102 'cloud':2,8,21,80,100 'dataflow':3,9,22,30,54,69,101 'defin':68 'detail':39 'distinct':57 'ecosystem':82 'exist':35 'featur':97 'fix':94 'follow':85 'gcp':103 'general':84 'get':37 'googl':1,7,20,79,99 'google-cloud':98 'google-cloud-dataflow-cli':6 'google-cloud-python':78 'interact':17 'interfac':51 'job':31,36,38,105 'job-manag':104 'librari':15,48 'list':34 'manag':29,106 'new':42,96 'one':43 'part':74 'pipelin':70 'programmat':28 'provid':49 'python':13,81 'receiv':89 'regular':90 'releas':87 'sdk':63 'servic':55 'submit':41 'templat':46 'updat':91 'use':66 'user':26","created_at":"2026-03-29T04:20:35.375561+00:00","updated_at":"2026-04-16T15:21:04.030485+00:00","problems":{"verify_error":"Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nImportError: cannot import name 'JobsClient' from 'google.cloud.dataflow_v1beta3' (/tmp/tmp_twtto88/venv/lib/python3.12/site-packages/google/cloud/dataflow_v1beta3/__init__.py)"},"ecosystem":"pypi","meta_description":null,"install_score":0,"quickstart_score":0,"quickstart_tag":"stale","pypi_latest":"0.14.0","cli_name":"","cli_version":null,"type":"library","homepage":"https://cloud.google.com/dataflow","github":"https://github.com/googleapis/google-cloud-python","docs":null,"changelog":null,"pypi":"https://pypi.org/project/google-cloud-dataflow-client/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["gcp","data"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"import_fail","verified_at":"2026-07-03","last_verified":"2026-07-03","next_check":"2026-07-10","install_tag":"stale"}}