diff --git a/data/nav.yml b/data/nav.yml index ecb46aad7a..26d791ad9c 100644 --- a/data/nav.yml +++ b/data/nav.yml @@ -725,6 +725,8 @@ path: apis/graphql/cookbooks/clusters - name: GitHub rate limits path: apis/graphql/cookbooks/github-rate-limits + - name: Hosted agents + path: apis/graphql/cookbooks/hosted-agents - name: Jobs path: apis/graphql/cookbooks/jobs - name: Pipelines diff --git a/pages/apis/descriptions/_common_create_cluster_fields.md b/pages/apis/descriptions/_common_create_cluster_fields.md index a46cc954f6..30b3a96f4b 100644 --- a/pages/apis/descriptions/_common_create_cluster_fields.md +++ b/pages/apis/descriptions/_common_create_cluster_fields.md @@ -7,4 +7,4 @@ - `color` (optional) provides the background color for this emoji and uses hex code syntax (for example, `#FFE0F1`). > 📘 A default queue is not automatically created -> Unlike creating a new cluster through the [Buildkite interface](#create-a-cluster-using-the-buildkite-interface), a default queue is not automatically created using this API call. To create a new/default queue for any new cluster created through an API call, you need to manually [create a new queue](/docs/pipelines/clusters/manage-queues#create-a-queue). +> Unlike creating a new cluster through the [Buildkite interface](#create-a-cluster-using-the-buildkite-interface), a default queue is not automatically created using this API call. To create a new/default queue for any new cluster created through an API call, you need to manually [create a new queue](/docs/pipelines/clusters/manage-queues#create-a-self-hosted-queue). diff --git a/pages/apis/graphql/cookbooks/hosted_agents.md b/pages/apis/graphql/cookbooks/hosted_agents.md new file mode 100644 index 0000000000..4a4ac4cae3 --- /dev/null +++ b/pages/apis/graphql/cookbooks/hosted_agents.md @@ -0,0 +1,98 @@ +# Hosted agents + +A collection of common tasks with [Hosted agents](/docs/pipelines/hosted-agents) using the GraphQL API. + +## Create a Buildkite hosted queue + +```graphql +mutation { + clusterQueueCreate( + input: { + organizationId: "organization-id" + clusterId: "cluster-id" + key: "hosted_linux_small" + description: "Small AMD64 Linux agents hosted by Buildkite." + hostedAgents: { + instanceShape: LINUX_AMD64_2X4 + } + } + ) { + clusterQueue { + id + uuid + key + description + dispatchPaused + hosted + hostedAgents { + instanceShape { + name + size + vcpu + memory + } + } + createdBy { + id + uuid + name + email + avatar { + url + } + } + } + } +} +``` + +Creates a small Buildkite hosted queue using AMD64-based Linux Buildkite hosted agents. The `instanceShape` value is referenced from the [InstanceShape](/docs/apis/graphql/schemas/enum/hostedagentinstanceshapename) enum, and represents the combination of machine type, architecture, CPU and Memory available to each job running on a hosted queue. The `LINUX_AMD64_2X4` value is a Linux AMD64 2 vCPU and 4 GB memory instance. + +Learn more about the instance shapes available for [Linux](#instance-shape-values-for-linux) and [Mac](#instance-shape-values-for-mac) Buildkite hosted agents. + +## Change the instance shape of a Buildkite hosted queue's agents + +```graphql +mutation { + clusterQueueUpdate( + input: { + organizationId: "organization-id" + id: "cluster-queue-id" + hostedAgents: { + instanceShape: LINUX_AMD64_4X16 + } + } + ) { + clusterQueue { + id + hostedAgents { + instanceShape { + name + size + vcpu + memory + } + } + } + } +} +``` + +To increase the size of the AMD64-based Linux agent instances for a Buildkite hosted queue, update the `instanceShape` value to a one of a greater size, such as `LINUX_AMD64_4X8`, which is a 4 vCPU and 8 GB memory. This allows you to scale the resources available to each job running on this Buildkite hosted queue. + +Learn more about the instance shapes available for [Linux](#instance-shape-values-for-linux) and [Mac](#instance-shape-values-for-mac) Buildkite hosted agents. + +> 📘 +> It is only possible to change the _size_ of the current instance shape assigned to this queue. It is not possible to change the current instance shape's machine type (from macOS to Linux, or vice versa), or for a Linux machine, its architecture (from AMD64 to ARM64, or vice versa). + +## Instance shape values for Linux + +Specify the appropriate **Instance shape** for the `instanceShape` value in your GraphQL API mutation. + +<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_linux' %> + +## Instance shape values for Mac + +Specify the appropriate **Instance shape** for the `instanceShape` value in your GraphQL API mutation. + +<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_mac' %> diff --git a/pages/apis/graphql/graphql_cookbook.md b/pages/apis/graphql/graphql_cookbook.md index 3f4e9f64cf..a781a881c9 100644 --- a/pages/apis/graphql/graphql_cookbook.md +++ b/pages/apis/graphql/graphql_cookbook.md @@ -12,6 +12,7 @@ There are recipes for a range of different topics, including: - [Builds](/docs/apis/graphql/cookbooks/builds) - [Clusters](/docs/apis/graphql/cookbooks/clusters) - [GitHub rate limits](/docs/apis/graphql/cookbooks/github-rate-limits) +- [Hosted agents](/docs/apis/graphql/cookbooks/hosted-agents) - [Jobs](/docs/apis/graphql/cookbooks/jobs) - [Pipelines](/docs/apis/graphql/cookbooks/pipelines) - [Pipeline templates](/docs/apis/graphql/cookbooks/pipeline-templates) diff --git a/pages/apis/rest_api/clusters.md b/pages/apis/rest_api/clusters.md index b4d5e82760..8adeb76f82 100644 --- a/pages/apis/rest_api/clusters.md +++ b/pages/apis/rest_api/clusters.md @@ -350,7 +350,7 @@ Required scope: `read_clusters` Success response: `200 OK` -### Create a queue +### Create a self-hosted queue ```bash curl -H "Authorization: Bearer $TOKEN" \ @@ -392,11 +392,91 @@ Required [request body properties](/docs/api#request-body-properties): +Required scope: `write_clusters` + +Success response: `201 Created` + +Error responses: + + + + + +
422 Unprocessable Entity{ "message": "Validation failed: Reason for failure" }
+ +### Create a Buildkite hosted queue + +```bash +curl -H "Authorization: Bearer $TOKEN" \ + -X POST "https://api.buildkite.com/v2/organizations/{org.slug}/clusters/{cluster.id}/queues" \ + -H "Content-Type: application/json" \ + -d '{ "key": "default", "description": "Queue of hosted Buildkite agents", "hostedAgents": { "instanceShape": "LINUX_AMD64_2X4" } }' +``` + +```json +{ + "id": "01885682-55a7-44f5-84f3-0402fb452e66", + "graphql_id": "Q2x1c3Rlci0tLTQyZjFhN2RhLTgxMmQtNDQzMC05M2Q4LTFjYzdjMzNhNmJjZg==", + "key": "default", + "description": "Queue of hosted Buildkite agents", + "url": "http://api.buildkite.com/v2/organizations/test/clusters/42f1a7da-812d-4430-93d8-1cc7c33a6bcf/queues/01885682-55a7-44f5-84f3-0402fb452e66", + "web_url": "http://buildkite.com/organizations/test/clusters/42f1a7da-812d-4430-93d8-1cc7c33a6bcf/queues/01885682-55a7-44f5-84f3-0402fb452e66", + "cluster_url": "http://api.buildkite.com/v2/organizations/test/clusters/42f1a7da-812d-4430-93d8-1cc7c33a6bcf", + "dispatch_paused": false, + "dispatch_paused_by": null, + "dispatch_paused_at": null, + "dispatch_paused_note": null, + "created_at": "2023-05-03T04:17:55.867Z", + "created_by": { + "id": "0187dfd4-92cf-4b01-907b-1146c8525dde", + "graphql_id": "VXNlci0tLTAxODdkZmQ0LTkyY2YtNGIwMS05MDdiLTExNDZjODUyNWRkZQ==", + "name": "Sam Kim", + "email": "sam@example.com", + "avatar_url": "https://www.gravatar.com/avatar/example", + "created_at": "2023-05-03T04:17:43.118Z" + }, + "hosted": true, + "hosted_agents": { + "instance_shape": { + "machine_type": "linux", + "architecture": "amd64", + "cpu": 2, + "memory": 4, + "name": "LINUX_AMD64_2X4" + } + } +} +``` + +Required [request body properties](/docs/api#request-body-properties): + + + + +
keyKey for the queue.
Example: "default" +
+ Optional [request body properties](/docs/api#request-body-properties): + + +
descriptionDescription for the queue.
Example: "The default queue for this cluster" +
hostedAgents + Configures this queue to use Buildkite hosted agents, along with its instance shape. This makes the queue a Buildkite hosted queue. +
+ Example: +
+ + { "instanceShape": "LINUX_AMD64_2X4" } + +
+ instanceShape (required when hostedAgents is specified): Describes the machine type, architecture, CPU, and RAM to provision for Buildkite hosted agent instances running jobs in this queue. +
+ Learn more about the instance shapes available for Linux and Mac hosted agents. +
@@ -451,6 +531,23 @@ curl -H "Authorization: Bearer $TOKEN" \ + + +
descriptionDescription for the queue.
Example: "The default queue for this cluster" +
hostedAgents + Configures this queue to use Buildkite hosted agents, along with its instance shape. This makes the queue a Buildkite hosted queue. +
+ Example: +
+ + { "instanceShape": "LINUX_AMD64_2X4" } + +
+ instanceShape (required when hostedAgents is specified): Describes the machine type, architecture, CPU, and RAM to provision for Buildkite hosted agent instances running jobs in this queue. +
+ It is only possible to change the size of the current instance shape assigned to this queue. It is not possible to change the current instance shape's machine type (from macOS to Linux, or vice versa), or for a Linux machine, its architecture (from AMD64 to ARM64, or vice versa).
+ Learn more about the instance shapes available for Linux and Mac Buildkite hosted agents. +
@@ -599,6 +696,18 @@ Error responses: +### Instance shape values for Linux + +Specify the appropriate **Instance shape** for the `instanceShape` value in your REST API call. + +<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_linux' %> + +### Instance shape values for Mac + +Specify the appropriate **Instance shape** for the `instanceShape` value in your REST API call. + +<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_mac' %> + ## Agent tokens An agent token is used to [connect agents to a cluster](/docs/pipelines/clusters/manage-clusters#connect-agents-to-a-cluster). diff --git a/pages/pipelines/clusters.md b/pages/pipelines/clusters.md index 679b631ae6..244f99271b 100644 --- a/pages/pipelines/clusters.md +++ b/pages/pipelines/clusters.md @@ -40,7 +40,7 @@ Learn more about working with clusters in [Manage clusters](/docs/pipelines/clus The most common queue attributes are based on infrastructure set-ups, such as: - Architecture (x86, arm64, Apple silicon, etc.) -- Size of agents (small, medium, large) +- Size of agents (small, medium, large, extra large) - Type of machine (macOS, Linux, Windows, GPU, etc.) Therefore, an example queue would be `small_mac_silicon`. diff --git a/pages/pipelines/clusters/manage_clusters.md b/pages/pipelines/clusters/manage_clusters.md index de374cba9c..cf93bdf493 100644 --- a/pages/pipelines/clusters/manage_clusters.md +++ b/pages/pipelines/clusters/manage_clusters.md @@ -238,9 +238,9 @@ where: ## Manage maintainers on a cluster -Buildkite administrators or users with the [_change organization_ permission](/docs/platform/team-management/permissions) can create clusters. +A user who is a [_Buildkite organization administrator_](/docs/platform/team-management/permissions#manage-teams-and-permissions-organization-level-permissions) can create clusters. Learn more about user permissions in Buildkite from [User and team permissions](/docs/platform/team-management/permissions). -As one of these types of users, you can add and manage other users or teams in your Buildkite organization as _maintainers_ of a cluster in the organization. A cluster maintainer can: +As a Buildkite organization administrator, you can add and manage other users or teams in your Buildkite organization as _maintainers_ of a cluster in the organization. A cluster maintainer can: - Update or delete the cluster. - Manage [agent tokens](/docs/agent/v3/tokens) associated with the cluster. diff --git a/pages/pipelines/clusters/manage_queues.md b/pages/pipelines/clusters/manage_queues.md index ad57b2a364..a8dad7b35f 100644 --- a/pages/pipelines/clusters/manage_queues.md +++ b/pages/pipelines/clusters/manage_queues.md @@ -6,33 +6,41 @@ This page provides details on how to manage queues within a [cluster](/docs/pipe When a new Buildkite organization is created, along with the automatically created [default cluster](/docs/pipelines/clusters/manage-clusters#setting-up-clusters) (named **Default cluster**), a default queue (named **default-queue**) within this cluster is also created. -A cluster can be configured with multiple queues, each of which can be used to represent a specific combination of your build/agent infrastructure, based on: +A cluster can be configured with multiple queues, each of which can be used to represent a specific combination of your [build/agent infrastructure](#agent-infrastructure), based on: - Architecture (x86-64, arm64, Apple silicon, etc.) -- Size of agents (small, medium, large) +- Size of agents (small, medium, large, extra large) - Type of machine (Mac, Linux, Windows, etc.) -Some example queues might be `mac_medium_x86`, `mac_large_silicon`, etc. +Some example queues might be `linux_medium_x86`, `mac_large_silicon`, etc. Having individual queues according to these breakdowns allows you to scale a set of similar agents, which Buildkite can then report on. -### Agent infrastructure +## Agent infrastructure -As part of setting up a queue, you can choose between setting up your agents using either [hosted](/docs/pipelines/hosted-agents) or self-hosted infrastructure. +Buildkite provides support for managing [Buildkite Agents](/docs/agent/v3) in your own self-hosted infrastructure, as well as a [Buildkite hosted infrastructure](/docs/pipelines/hosted-agents) for managing these agents. -Buildkite provides a hosted infrastructure for your [Buildkite Agents](/docs/agent/v3), as well as support for self-hosted infrastructure, where you provide the infrastructure that hosts Buildkite Agents. +When setting up a queue, you can choose between configuring it with Buildkite Agents running in either of these types of infrastructure. -> 📘 -> Creating [Buildkite hosted agent](/docs/pipelines/hosted-agents) queues is currently only supported through the [Buildkite interface](#create-a-queue-using-the-buildkite-interface). It is not possible to create hosted agent queues using the [REST](#create-a-queue-using-the-rest-api) or [GraphQL](#create-a-queue-using-the-graphql-api) APIs. +Learn more about how to set up and create a queue using either self-hosted agents (known as a [self-hosted queue](#create-a-self-hosted-queue)) or Buildkite hosted agents (known as a [Buildkite hosted queue](#create-a-buildkite-hosted-queue)). -## Create a queue +Be aware that it is not possible to create a queue that uses a mix of self-hosted and Buildkite hosted agents. If you do need to use a combination of these different agent types for your pipeline builds, create separate self-hosted and Buildkite hosted queues for these agents and use [agent or queue tags](/docs/agent/v3/queues#setting-an-agents-queue), or a combination of both, to target the appropriate queues. -New queues can be created by a [cluster maintainer](/docs/pipelines/clusters/manage-clusters#manage-maintainers-on-a-cluster) using the [**Queues** page of a cluster](#create-a-queue-using-the-buildkite-interface), as well as the [REST API's](#create-a-queue-using-the-rest-api) or [GraphQL API's](#create-a-queue-using-the-graphql-api) create a queue feature. +Furthermore, once a queue has been created, it is not possible to change its type from a self-hosted to a Buildkite hosted queue, or vice versa. If you do need to change your type of agent infrastructure, use a queue with the appropriate hosted queue type, or create a new queue to suit your new agent infrastructure. + +## Create a self-hosted queue + +Self-hosted queues use [Buildkite Agents installed in your own infrastructure](/docs/agent/v3/installation) to run your pipeline builds. New self-hosted queues can be created by a [cluster maintainer](/docs/pipelines/clusters/manage-clusters#manage-maintainers-on-a-cluster) using the [Buildkite interface](#create-a-self-hosted-queue-using-the-buildkite-interface), as well as the [REST API's](#create-a-self-hosted-queue-using-the-rest-api) or [GraphQL API's](#create-a-self-hosted-queue-using-the-graphql-api) create a queue feature. For these API requests, the _cluster ID_ value submitted in the request is the target cluster the queue will be created in. When you [create a new cluster](/docs/pipelines/clusters/manage-clusters#create-a-cluster) through the [Buildkite interface](/docs/pipelines/clusters/manage-clusters#create-a-cluster-using-the-buildkite-interface), this cluster automatically has an initial **default** queue. +Multiple self-hosted agents can connect to your self-hosted queue by ensuring that the agent is configured to use both of the following: + +- The [cluster's agent token](/docs/agent/v3/tokens#using-and-storing-tokens) +- The [agent tag](/docs/agent/v3/queues#setting-an-agents-queue) targeting your self-hosted queue + ### Using the Buildkite interface To create a new queue using the Buildkite interface: @@ -42,26 +50,22 @@ To create a new queue using the Buildkite interface: 1. On the **Queues** page, select **New Queue** to open the **Create a new Queue** page. 1. In the **Create a key** field, enter a unique _key_ for the queue, which can only contain letters, numbers, hyphens, and underscores, as valid characters. 1. Select the **Add description** checkbox to enter an optional longer description for the queue. This description appears under the queue's key, which is listed on the **Queues** page, as well as when viewing the queue's details. -1. In the **Select your agent infrastructure** section, select between [**Hosted**](/docs/pipelines/hosted-agents) or **Self hosted** for your agent infrastructure. -1. If you chose **Hosted**, complete the remaining sub-steps: - 1. In the new **Configure your hosted agent infrastructure** section, select your **Machine type** ([**Linux**](/docs/pipelines/hosted-agents/linux) or [**macOS**](/docs/pipelines/hosted-agents/mac)). - 1. If you selected **Linux**, within **Architecture**, you can choose between **AMD64** (the default and recommended) or **ARM64** architectures for the Linux machines running as hosted agents. To switch to **ARM64**, select **Change**, followed by **ARM64 (AArch64)**. - 1. Select the appropriate **Capacity** for your hosted agent machine type (**Small**, **Medium** or **Large**). Take note of the additional information provided in the new **Hosted agents trial** section, which changes based on your selected **Capacity**. +1. In the **Select your agent infrastructure** section, select **Self hosted** for your agent infrastructure. 1. Select **Create Queue**. The new queue's details are displayed, indicating the queue's key and its description (if configured) underneath this key. Select **Queues** on the interface again to list all configured queues in your cluster. ### Using the REST API -To [create a new self-hosted agent queue](/docs/apis/rest-api/clusters#queues-create-a-queue) using the [REST API](/docs/apis/rest-api), run the following example `curl` command: +To [create a new self-hosted agent queue](/docs/apis/rest-api/clusters#queues-create-a-self-hosted-queue) using the [REST API](/docs/apis/rest-api), run the following example `curl` command: ```curl curl -H "Authorization: Bearer $TOKEN" \ -X POST "https://api.buildkite.com/v2/organizations/{org.slug}/clusters/{cluster.id}/queues" \ -H "Content-Type: application/json" \ -d '{ - "key": "mac_large_silicon", - "description": "The queue for powerful macOS agents running on Apple silicon architecture." + "key": "linux_small_amd", + "description": "A small self-hosted AMD64 Linux agent." }' ``` @@ -85,8 +89,114 @@ mutation { input: { organizationId: "organization-id" clusterId: "cluster-id" - key: "mac_large_silicon" - description: "The queue for powerful macOS agents running on Apple silicon architecture." + key: "linux_small_amd" + description: "A small self-hosted AMD64 Linux agent." + } + ) { + clusterQueue { + id + uuid + key + description + dispatchPaused + hosted + createdBy { + id + uuid + name + email + avatar { + url + } + } + } + } +} +``` + +where: + +<%= render_markdown partial: 'apis/descriptions/graphql_organization_id' %> + +<%= render_markdown partial: 'apis/descriptions/graphql_cluster_id' %> + +<%= render_markdown partial: 'apis/descriptions/common_create_queue_fields' %> + +## Create a Buildkite hosted queue + +Buildkite hosted queues use [Buildkite's hosted agent infrastructure](/docs/pipelines/hosted-agents) to run your pipeline builds. New Buildkite hosted queues can be created by a [cluster maintainer](/docs/pipelines/clusters/manage-clusters#manage-maintainers-on-a-cluster) using the [Buildkite interface](#create-a-buildkite-hosted-queue-using-the-buildkite-interface), the [REST API](#create-a-buildkite-hosted-queue-using-the-rest-api), or the [GraphQL API](#create-a-buildkite-hosted-queue-using-the-graphql-api). + +When you create a Buildkite hosted queue, you can choose the machine type (Linux or macOS) and the capacity (small, medium, large, or extra large), known as the _instance shape_, of the Buildkite hosted agents that will run your builds. + +Only one instance shape can be configured on a Buildkite hosted queue. However, depending on your pipeline's requirements, multiple Buildkite hosted agents of the queue's configured instance shape can be spawned automatically by Buildkite. + +### Using the Buildkite interface + +To create a new Buildkite hosted queue using the Buildkite interface: + +1. Select **Agents** in the global navigation to access the **Clusters** page. +1. Select the cluster in which to create the new queue. +1. On the **Queues** page, select **New Queue** to open the **Create a new Queue** page. +1. In the **Create a key** field, enter a unique _key_ for the queue, which can only contain letters, numbers, hyphens, and underscores, as valid characters. +1. Select the **Add description** checkbox to enter an optional longer description for the queue. This description appears under the queue's key, which is listed on the **Queues** page, as well as when viewing the queue's details. +1. In the **Select your agent infrastructure** section, select **Hosted** for your agent infrastructure. +1. In the new **Configure your hosted agent infrastructure** section, select your **Machine type** ([**Linux**](/docs/pipelines/hosted-agents/linux) or [**macOS**](/docs/pipelines/hosted-agents/mac)). +1. If you selected **Linux**, within **Architecture**, you can choose between **AMD64** (the default and recommended) or **ARM64** architectures for the Linux machines running as hosted agents. To switch to **ARM64**, select **Change**, followed by **ARM64 (AArch64)**. +1. Select the appropriate **Capacity** for your hosted agent machine type (**Small**, **Medium** or **Large**). Take note of the additional information provided in the new **Hosted agents trial** section, which changes based on your selected **Capacity**. +1. Select **Create Queue**. + + The new queue's details are displayed, indicating the queue's key and its description (if configured) underneath this key. Select **Queues** on the interface again to list all configured queues in your cluster. + +### Using the REST API + +To [create a new Buildkite hosted queue](/docs/apis/rest-api/clusters#queues-create-a-buildkite-hosted-queue) using the [REST API](/docs/apis/rest-api), run the following example `curl` command: + +```curl +curl -H "Authorization: Bearer $TOKEN" \ + -X POST "https://api.buildkite.com/v2/organizations/{org.slug}/clusters/{cluster.id}/queues" \ + -H "Content-Type: application/json" \ + -d '{ + "key": "mac_small_silicon", + "description": "macOS agents running on Apple silicon architecture.", + "hostedAgents": { + "instanceShape": "MACOS_M2_4X7" + } + }' +``` + +where: + +<%= render_markdown partial: 'apis/descriptions/rest_access_token' %> + +<%= render_markdown partial: 'apis/descriptions/rest_org_slug' %> + +<%= render_markdown partial: 'apis/descriptions/rest_cluster_id' %> + +<%= render_markdown partial: 'apis/descriptions/common_create_queue_fields' %> + +- `hostedAgents` (required) configures this queue to use [Buildkite hosted agents](/docs/pipelines/hosted-agents), which makes this a _Buildkite hosted queue_, and defines the instance shape (within its `instanceShape` object) for this queue's [Linux-](#create-a-buildkite-hosted-queue-instance-shape-values-for-linux) or [Mac-](#create-a-buildkite-hosted-queue-instance-shape-values-for-mac)based Buildkite hosted agent. For example: + + ```json + "hostedAgents": { + "instanceShape": "LINUX_AMD64_2X4" + } + ``` + +### Using the GraphQL API + +To [create a new Buildkite hosted queue](/docs/apis/graphql/schemas/mutation/clusterqueuecreate) using the [GraphQL API](/docs/apis/graphql-api), run the following example mutation: + +```graphql +mutation { + clusterQueueCreate( + input: { + organizationId: "organization-id" + clusterId: "cluster-id" + key: "mac_small_silicon" + description: "macOS agents running on Apple silicon architecture." + hostedAgents: { + instanceShape: MACOS_M2_4X7 + } } ) { clusterQueue { @@ -95,6 +205,15 @@ mutation { key description dispatchPaused + hosted + hostedAgents { + instanceShape { + name + size + vcpu + memory + } + } createdBy { id uuid @@ -117,6 +236,26 @@ where: <%= render_markdown partial: 'apis/descriptions/common_create_queue_fields' %> +- `hostedAgents` (required) configures this queue to use [Buildkite hosted agents](/docs/pipelines/hosted-agents), which makes this a _Buildkite hosted queue_, and defines the instance shape (within its `instanceShape` object) for this queue's [Linux-](#create-a-buildkite-hosted-queue-instance-shape-values-for-linux) or [Mac-](#create-a-buildkite-hosted-queue-instance-shape-values-for-mac) based Buildkite hosted agent. For example: + + ```graphql + hostedAgents: { + instanceShape: LINUX_AMD64_2X4 + } + ``` + +### Instance shape values for Linux + +Specify the appropriate **Instance shape** for the `instanceShape` value in your API call. + +<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_linux' %> + +### Instance shape values for Mac + +Specify the appropriate **Instance shape** for the `instanceShape` value in your API call. + +<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_mac' %> + ## Pause and resume a queue You can pause a queue to prevent any jobs of the cluster's pipelines from being dispatched to agents associated with this queue. diff --git a/pages/pipelines/getting_started.md b/pages/pipelines/getting_started.md index dc51385648..cdf1277b19 100644 --- a/pages/pipelines/getting_started.md +++ b/pages/pipelines/getting_started.md @@ -53,11 +53,13 @@ You need at least one agent configured within its own queue and cluster to run b You can create the first [Buildkite hosted agent](/docs/pipelines/hosted-agents) within a Buildkite organization for a two-week free trial, after which a usage cost (based on the agent's capacity) is charged per minute. +Before creating your Buildkite hosted agent, ensure you have a [cluster](/docs/pipelines/clusters/manage-clusters) (for example, **Default cluster**) you can connect this agent to. + To create a hosted agent: 1. Navigate to the [cluster](/docs/pipelines/clusters/manage-clusters) you want to run your pipeline in. To do this, select **Agents** in the global navigation to access the **Clusters** page. 1. Select the cluster (for example, **Default cluster**) to which the hosted agent will be added. -1. Follow the [Create a queue](/docs/pipelines/clusters/manage-queues#create-a-queue) > [Using the Buildkite interface](/docs/pipelines/clusters/manage-queues#create-a-queue-using-the-buildkite-interface) instructions to begin creating your hosted agent within its own queue. +1. Follow the [Create a Buildkite hosted queue](/docs/pipelines/clusters/manage-queues#create-a-buildkite-hosted-queue) > [Using the Buildkite interface](/docs/pipelines/clusters/manage-queues#create-a-buildkite-hosted-queue-using-the-buildkite-interface) instructions to begin creating your hosted agent within its own queue. As part of this process: * In the **Select your agent infrastructure** section, choose **Hosted**. diff --git a/pages/pipelines/hosted_agents.md b/pages/pipelines/hosted_agents.md index c9a4263c85..bc92e9508c 100644 --- a/pages/pipelines/hosted_agents.md +++ b/pages/pipelines/hosted_agents.md @@ -6,15 +6,15 @@ With hosted agents, Buildkite handles infrastructure management tasks, such as p ## Hosted agent types -Buildkite offers both [Mac](/docs/pipelines/hosted-agents/mac) and [Linux](/docs/pipelines/hosted-agents/linux) hosted agents. Refer to these respective pages for detailed information about available the sizes and configurations for these agents. +Buildkite offers both [Mac](/docs/pipelines/hosted-agents/mac) and [Linux](/docs/pipelines/hosted-agents/linux) hosted agents. Usage of all instance types is billed on a per-minute basis. Every Buildkite hosted agent within a cluster benefits from hypervisor-level isolation, ensuring robust separation between each instance. -## Creating a hosted agent queue +## Creating a Buildkite hosted queue -You can set up distinct hosted agent queues, each configured with specific types and sizes to efficiently manage jobs with varying requirements. +You can set up distinct queues for your Buildkite hosted agents (known as _Buildkite hosted queues_), each configured with a specific type and size of hosted agent, to efficiently manage jobs with varying requirements. Learn more about how to do this in [Create a Buildkite hosted queue](/docs/pipelines/clusters/manage-queues#create-a-buildkite-hosted-queue). For example you may have two queues set up: @@ -23,8 +23,9 @@ For example you may have two queues set up: Learn more about: -- Best practices for configuring queues in [How should I structure my queues](/docs/pipelines/clusters#clusters-and-queues-best-practices-how-should-i-structure-my-queues). -- How to set up and create a Buildkite hosted agent queue in [Manage queues](/docs/pipelines/clusters/manage-queues). +- Best practices for configuring queues in [How should I structure my queues](/docs/pipelines/clusters#clusters-and-queues-best-practices-how-should-i-structure-my-queues) of the [Clusters overview](/docs/pipelines/clusters). + +- Configuring queues in general, in [Manage queues](/docs/pipelines/clusters/manage-queues). ## Using GitHub repositories in your hosted agent pipelines diff --git a/pages/pipelines/hosted_agents/linux.md b/pages/pipelines/hosted_agents/linux.md index 7e9d9cc00c..e08d1a10e5 100644 --- a/pages/pipelines/hosted_agents/linux.md +++ b/pages/pipelines/hosted_agents/linux.md @@ -5,22 +5,15 @@ Linux instances for Buildkite hosted agents are offered with two architectures: - AMD64 (x64_86) - ARM64 (AArch64) -To accommodate different workloads, instances are capable of running up to 8 hours. If you require longer running agents please contact support. +To accommodate different workloads, instances are capable of running up to 8 hours. If you require longer running agents, please contact support at support@buildkite.com. -## Size +## Sizes -Buildkite offers a selection of instance sizes, allowing you to tailor your hosted agents' resources to the demands of your jobs. Below is a breakdown of the available sizes. +Buildkite offers a selection of Linux instance types (each based on a different combination of size and architecture, known as an _instance shape_), allowing you to tailor your hosted agent resources to the demands of your jobs. - - - - - - - - - -
SizevCPURAM
Small24 GB
Medium416 GB
Large832 GB
+<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_linux' %> + +Extra large instances are available on request. Please contact support@buildkite.com to have them enabled for your account. ## Cache volumes @@ -181,7 +174,7 @@ You can create an agent image: 1. Select **Agents** in the global navigation to access the **Clusters** page. 1. Select the cluster in which to create the new agent image. - **Note:** Before continuing, ensure you have created a hosted agent queue (based on Linux architecture) within this cluster. Learn more about how to do this in [Create a queue](/docs/pipelines/clusters/manage-queues#create-a-queue). + **Note:** Before continuing, ensure you have created a hosted agent queue (based on Linux architecture) within this cluster. Learn more about how to do this in [Create a Buildkite hosted queue](/docs/pipelines/clusters/manage-queues#create-a-buildkite-hosted-queue). 1. Select **Agent Images** to open the **Agent Images** page. 1. Select **New Image** to open the **New Agent Image** dialog. diff --git a/pages/pipelines/hosted_agents/mac.md b/pages/pipelines/hosted_agents/mac.md index f6dd6c9577..6a8188ade7 100644 --- a/pages/pipelines/hosted_agents/mac.md +++ b/pages/pipelines/hosted_agents/mac.md @@ -2,22 +2,15 @@ Mac instances for Buildkite hosted agents are only offered with [Apple silicon](https://en.wikipedia.org/wiki/Apple_silicon) architecture. Please contact support if you have specific needs for Intel machines. -To accommodate different workloads, instances are capable of running up to 4 hours. If you require longer running agents please contact support. - -## Size - -We offer a selection of instance sizes, allowing you to tailor your hosted agent resources to the demands of your jobs. Below is a breakdown of the available sizes. - - - - - - - - - - -
SizevCPURAM
Small47 GB
Medium614 GB
Large1228 GB
+To accommodate different workloads, instances are capable of running up to 4 hours. If you require longer running agents, please contact support at support@buildkite.com. + +## Sizes + +Buildkite offers a selection of Mac instance types (each based on a different size combination of virtual CPU power and memory capacity, known as an _instance shape_), allowing you to tailor your hosted agents' resources to the demands of your jobs. + +<%= render_markdown partial: 'shared/hosted_agents/hosted_agents_instance_shape_table_mac' %> + +Extra large instances are available on request. Please contact support@buildkite.com to have them enabled for your account. ## Mac instances software support diff --git a/pages/shared/hosted_agents/_hosted_agents_instance_shape_table_linux.md b/pages/shared/hosted_agents/_hosted_agents_instance_shape_table_linux.md new file mode 100644 index 0000000000..eac132afaa --- /dev/null +++ b/pages/shared/hosted_agents/_hosted_agents_instance_shape_table_linux.md @@ -0,0 +1,83 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Instance shapeSizeArchitecturevCPUMemory
+ LINUX_AMD64_2X4 + SmallAMD6424 GB
+ LINUX_AMD64_4X16 + MediumAMD64416 GB
+ LINUX_AMD64_8X32 + LargeAMD64832 GB
+ LINUX_AMD64_16X64 + Extra LargeAMD641664 GB
+ LINUX_ARM64_2X4 + SmallARM6424 GB
+ LINUX_ARM64_4X16 + MediumARM64416 GB
+ LINUX_ARM64_8X32 + LargeARM64832 GB
+ LINUX_ARM64_16X64 + Extra LargeARM641664 GB
diff --git a/pages/shared/hosted_agents/_hosted_agents_instance_shape_table_mac.md b/pages/shared/hosted_agents/_hosted_agents_instance_shape_table_mac.md new file mode 100644 index 0000000000..e8fd4dd12a --- /dev/null +++ b/pages/shared/hosted_agents/_hosted_agents_instance_shape_table_mac.md @@ -0,0 +1,42 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Instance shapeSizevCPUMemory
+ MACOS_M2_4X7 + Small47 GB
+ MACOS_M2_6X14 + Medium614 GB
+ MACOS_M2_12X28 + Large1228 GB
+ MACOS_M4_12X56 + Extra Large1256 GB
diff --git a/vale/styles/Buildkite/existence-case-sensitive.yml b/vale/styles/Buildkite/existence-case-sensitive.yml index 214e7c2c2f..9cacb4e89a 100644 --- a/vale/styles/Buildkite/existence-case-sensitive.yml +++ b/vale/styles/Buildkite/existence-case-sensitive.yml @@ -40,7 +40,7 @@ swap: (?i:GraphQL): GraphQL (?i:IronWorker): IronWorker (?i:iTerm): iTerm - (?i:Mac): Mac + (?i:(?