{"id":490,"library":"google-cloud-batch","title":"Google Cloud Batch","description":"The `google-cloud-batch` Python client library provides programmatic access to the Google Cloud Batch API, a fully managed service for running batch jobs at scale. It simplifies the orchestration of high-performance computing (HPC), AI/ML, and data processing workloads by handling infrastructure provisioning, scheduling, execution, and cleanup. The library is currently at version 0.20.0 and is part of the `google-cloud-python` monorepo, which typically sees frequent releases.","status":"active","version":"0.20.0","language":"python","source_language":"en","source_url":"https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-batch","tags":["google cloud","batch processing","serverless","job scheduling","hpc","ai/ml"],"install":[{"cmd":"pip install google-cloud-batch","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Core Google API client functionality.","package":"google-api-core"},{"reason":"Provides Pythonic wrappers for Protobuf messages.","package":"proto-plus"},{"reason":"Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data.","package":"protobuf"}],"imports":[{"wrong":"from google.cloud import batch_v1","symbol":"BatchServiceClient","correct":"from google.cloud.batch_v1 import BatchServiceClient"}],"quickstart":{"code":"import os\nfrom google.cloud import batch_v1\nfrom google.cloud.batch_v1 import types\n\ndef create_simple_container_job(\n    project_id: str,\n    region: str,\n    job_name: str,\n) -> types.Job:\n    \"\"\"Creates and runs a simple container job in Google Cloud Batch.\"\"\"\n    client = batch_v1.BatchServiceClient()\n\n    # Define what will be done as part of the job.\n    runnable = types.Runnable()\n    runnable.container = types.Runnable.Container(\n        image_uri=\"gcr.io/google-containers/busybox\",\n        entrypoint=\"/bin/sh\",\n        commands=[\n            \"-c\",\n            \"echo Hello world! This is task ${BATCH_TASK_INDEX}. This job has a total of ${BATCH_TASK_COUNT} tasks.\",\n        ],\n    )\n\n    # Jobs can be divided into tasks. In this case, we have one task group with one task.\n    task_spec = types.TaskSpec(runnables=[runnable])\n    task_group = types.TaskGroup(\n        task_spec=task_spec,\n        task_count=1,\n        parallelism=1,\n    )\n\n    # Policies for VM allocation.\n    # Using a general purpose machine type like 'e2-standard-4'.\n    # Ensure the specified region supports the machine type.\n    allocation_policy = types.AllocationPolicy(\n        instances=[\n            types.AllocationPolicy.InstancePolicyOrTemplate(\n                policy=types.AllocationPolicy.InstancePolicy(machine_type=\"e2-standard-4\")\n            ),\n        ],\n        location=types.AllocationPolicy.LocationPolicy(\n            allowed_locations=[f\"regions/{region}\"]\n        )\n    )\n\n    # Define the job itself.\n    job = types.Job(\n        name=job_name, # Name needs to be unique per project and region\n        task_groups=[task_group],\n        allocation_policy=allocation_policy,\n        labels={\n            \"environment\": \"dev\",\n            \"framework\": \"batch-quickstart\",\n        },\n        logs_policy=types.LogsPolicy(destination=types.LogsPolicy.Destination.CLOUD_LOGGING),\n    )\n\n    request = types.CreateJobRequest(\n        parent=f\"projects/{project_id}/locations/{region}\",\n        job_id=job_name,\n        job=job,\n    )\n\n    response = client.create_job(request=request)\n    print(f\"Job created: {response.name}\")\n    return response\n\nif __name__ == \"__main__\":\n    project_id = os.environ.get(\"GOOGLE_CLOUD_PROJECT\", \"your-gcp-project-id\")\n    region = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\") # Choose an available region\n    job_id = os.environ.get(\"BATCH_JOB_ID\", \"my-sample-batch-job-1\") # Unique ID for the job\n\n    if project_id == \"your-gcp-project-id\":\n        print(\"Please set the GOOGLE_CLOUD_PROJECT environment variable or replace 'your-gcp-project-id'.\")\n    elif region == \"us-central1\":\n        print(\"Consider setting the GOOGLE_CLOUD_REGION environment variable or choose a different region.\")\n    else:\n        try:\n            created_job = create_simple_container_job(project_id, region, job_id)\n            print(f\"Monitor job in console: https://console.cloud.google.com/batch/jobs/{region}/{job_id}?project={project_id}\")\n        except Exception as e:\n            print(f\"Error creating job: {e}\")\n            print(\"Ensure the Batch API is enabled and your service account has 'Batch Job Editor' (roles/batch.jobs.editor) or equivalent permissions.\")\n","lang":"python","description":"This quickstart demonstrates how to create a basic Google Cloud Batch job that runs a simple 'Hello World' container image. Ensure the Google Cloud Batch API is enabled for your project, and your environment is authenticated with Application Default Credentials (e.g., via `gcloud auth application-default login`). The example uses environment variables for project ID, region, and job ID for easy customization."},"warnings":[{"fix":"Refer to the official changelog (https://cloud.google.com/python/docs/release-notes/all) for each new minor or patch release and review any breaking changes. Pin your dependency versions to specific patch releases to manage updates carefully.","message":"As a pre-GA (0.x.x) client library, the API surface and underlying RPCs of `google-cloud-batch` are subject to backward-incompatible changes without a major version bump. This means updates might introduce breaking changes to existing code.","severity":"breaking","affected_versions":"0.x.x (all versions before 1.0.0)"},{"fix":"For local development, use `gcloud auth application-default login`. For deployment on GCP services (Compute Engine, Cloud Run, Cloud Functions), leverage the attached service account. For external workloads, consider Workload Identity Federation. Do not commit service account keys to version control.","message":"Authentication with Google Cloud client libraries often relies on Application Default Credentials (ADC). Hardcoding service account key JSON files directly into applications is a common anti-pattern and security risk.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure the service account creating the job has `roles/batch.jobs.editor` or equivalent. For jobs using custom service accounts, ensure the caller has `iam.serviceAccounts.actAs` permission on that service account. Check Compute Engine quotas in your project and region, and request increases if necessary.","message":"Batch job creation can fail due to insufficient IAM permissions (e.g., `iam.serviceAccounts.actAs`) for the service account used by the job or due to insufficient resource quotas in the specified region.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always use the latest available Compute Engine VM OS images or ensure custom images are based on up-to-date kernels. Monitor Batch API release notes for known issues related to VM images.","message":"Jobs might fail if they specify Compute Engine (or custom) VM OS images with outdated kernels. This can lead to unexpected job failures.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Explicitly configure Python's `logging` module to handle logs from `google.cloud.batch`. Be mindful of log destinations and access restrictions if sensitive data might be logged. You can also use the `GOOGLE_SDK_PYTHON_LOGGING_SCOPE` environment variable for simple configuration.","message":"The client library's internal logging can be verbose and may contain sensitive information. By default, logging events from the library are not handled.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Set the `GOOGLE_CLOUD_PROJECT` environment variable in your environment, or explicitly provide the project ID as a parameter to the client library constructor or relevant method (e.g., `project='your-gcp-project-id'`).","message":"The client library failed to retrieve a Google Cloud project ID. This often happens if the `GOOGLE_CLOUD_PROJECT` environment variable is not set, or the project ID is not passed directly to the client.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure the `GOOGLE_CLOUD_PROJECT` environment variable is set. Alternatively, configure `gcloud` with `gcloud config set project [PROJECT_ID]` or pass the `project` argument explicitly to the client constructor, e.g., `batch_client = batch_v1.BatchServiceClient(project='your-project-id')`.","message":"Google Cloud client libraries require a target Google Cloud project to operate. Failing to specify the project ID (e.g., via `GOOGLE_CLOUD_PROJECT` environment variable, `gcloud` configuration, or explicit client constructor arguments) will prevent successful API calls.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"search_vec":"'0.20.0':60 'access':14 'ai/ml':41,84 'api':20 'batch':3,8,19,27,78 'cleanup':53 'client':10 'cloud':2,7,18,68,77 'comput':39 'current':57 'data':43 'execut':51 'frequent':74 'fulli':22 'googl':1,6,17,67,76 'google-cloud-batch':5 'google-cloud-python':66 'handl':47 'high':37 'high-perform':36 'hpc':40,83 'infrastructur':48 'job':28,81 'librari':11,55 'manag':23 'monorepo':70 'orchestr':34 'part':63 'perform':38 'process':44,79 'programmat':13 'provid':12 'provis':49 'python':9,69 'releas':75 'run':26 'scale':30 'schedul':50,82 'see':73 'serverless':80 'servic':24 'simplifi':32 'typic':72 'version':59 'workload':45","created_at":"2026-03-28T15:15:51.027719+00:00","updated_at":"2026-04-16T15:19:49.252275+00:00","problems":null,"ecosystem":"pypi","meta_description":null,"install_score":100,"quickstart_score":80,"quickstart_tag":"verified","pypi_latest":"0.22.0","cli_name":"","cli_version":null,"type":"library","homepage":"https://cloud.google.com/batch","github":"https://github.com/googleapis/google-cloud-python","docs":null,"changelog":null,"pypi":"https://pypi.org/project/google-cloud-batch/","npm":null,"openapi_spec":null,"status_page":null,"smithery":null,"categories":["gcp"],"base_url":null,"auth_type":null,"provenance":{"verified_status":"passing","verified_at":"2026-07-03","last_verified":"2026-07-03","next_check":"2026-08-02","install_tag":"verified"}}