BigQuery

​

Extra information

System documentation: BigQuery API Reference

​

Connector package documentation

​

Table of Contents

• 1. Setting up HTTP Client BigQuery
  • 1.1 Steps
• 2. Subscribing to entities from BigQuery
  • 2.1 Steps
  • 2.2 General concept
• 3. Publishing entity to BigQuery
  • 3.1 Steps
  • 3.2 General concept

​

1. Setting up HTTP Client BigQuery

1.1 Steps

1. In the Alumio Dashboard, go to Clients -> HTTP Clients. Create a new HTTP Client.
2. Select BigQuery HTTP Client as Platform and click Next Step
3. Fill in the Base URI and then select OAuth 2 for the Authentication type or Bearer for the alternative.
4. Fill in the Client ID and the Client Secret (See the BigQuery Authentication Documentation for more information)
5. You can optionally enable logging of requests to get the log of the authentication.
6. Click next step, fill in the name of the HTTP Client, and finally, you can save the HTTP Client for use.

​

2. Subscribing to Entities from BigQuery

2.1 Steps

1. Go to Connections -> Incoming and create a new incoming configuration and select “BigQuery Subscriber” as the subscriber.
2. Select the entity you want to subscribe to from BigQuery.
3. Add any request parameters needed to subscribe to the entity.
4. Select the BigQuery HTTP Client to use.
5. You can optionally add a transformer to transform the request parameters into something you will request from BigQuery.
6. You can also optionally enable pagination to allow Alumio to fetch paginated entities.

2.2 General Concept We follow BigQuery documentation on building this connector, so you can easily use the connector based on BigQuery API Reference. Entity The entity you want to subscribe to BigQuery. Please refer to the entities from BigQuery API Reference page. Request Parameters These are the parameters you can provide to get the needed entity based on the BigQuery API Reference. Please look at the example below on how to fill request parameters when we want to run a BigQuery SQL query synchronously and return query results (Queries). From the endpoint, we can see that we need to set the “query” as a payload. Alumio will then make a POST request to BigQuery with the URL /queries

• Path is the path parameter of the URL. For example, the {id}.
• A query is defined set of parameters attached to the end of a URL.
  Example: www.url.com?type=true&start=0

HTTP Client The HTTP Client configuration you will use to access BigQuery. 3.1 Steps

1. Go to Connections -> Outgoing, create a new outgoing configuration and select “BigQuery Publisher” as the publisher.
2. Select the Method that you want the publisher to perform with the data.
3. Select the BigQuery entity you want the data to be published to.
4. You can optionally add any query data to the request on the key query.
5. You can optionally add any payload data to the request on the key payload.
6. Select the HTTP Client to use.

3.2 General Concept Like the subscriber, the publisher has similar methods on how to use it. The difference is the payload that will be submitted by the publisher originating from the data (from routes, from transformers in outgoing configuration), though you can also add or modify the payload using the Request Transformer. Method Currently, we support the POST method, PUT, PATCH method, and DELETE method for submitting data to BigQuery. Entity We support publishing to all BigQuery endpoints. Please refer to BigQuery API Reference page. For example, we want to patch the dataset information. Please select “Patch” as the Method and “Patch Dataset”. From the documentation, we can see that we need to set a path for “datasetId” and body parameters. That is why we set the “datasetId” inside the path key and the body parameters inside the payload key in the data. Alumio will send an API request to BigQuery with URL /dataset/_datasetId12334545465

• Path is the path parameter of the URL. For example, the {id}.
• A query isa defined set of parameters attached to the end of a URL.
  Example: www.url.com?type=true&start=0
• Payload is the data contained within a request.

HTTP Client The HTTP Client configuration you will use to access BigQuery.