Google Bigtable is a distributed, column-oriented data store created by Google Inc. to handle very large amounts of structured data associated with the company's Internet search and Web services operations. Bigtable was designed to support applications requiring massive scalability; from its first iteration, the technology was intended to be used with petabytes of data. The database was designed to be deployed on clustered systems and uses a simple data model that Google has described as "a sparse, distributed, persistent multidimensional sorted map." Data is assembled in order by row key, and indexing of the map is arranged according to row, column keys and timestamps. Compression algorithms help achieve high capacity.

Connect Google Bigtable with LogicHub

Google BigTable Authentication

We can access Google BigTable using Google service account credentials. Below details are required for that.

1. Google service account credentials file. This JSON file can be created/Downloaded from https://console.cloud.google.com/iam-admin/serviceaccounts

2. Navigate to Automations > Integrations.

3. Search for Google Bigtable.

4. Click Details, then the + icon. Enter the required information in the following fields.

   • Label: Enter a connection name.
   • Reference Values: Define variables here to templatize integration connections and actions. For example, you can use https://www.{{hostname}}.com where, hostname is a variable defined in this input. For more information on how to add data, see 'Add Data' Input Type for Integrations.
   • Verify SSL: Select option to verify connecting server's SSL certificate (Default is Verify SSL Certificate).
   • Remote Agent: Run this integration using the LogicHub Remote Agent.
   • JSON Credentials: Copy the content of the Google credentials JSON file that you would like to use.

5. After you've entered all the details, click Connect.

Actions with Google BigTable:

Create Table

Create a table in Google Bigtable, Since this is noSQL we don’t need to define the schema. We need two inputs to create a table.

• Instance Name: Name of your Google BigTable Instance.
• Table Name: Name of the table we wish to create in the above instance.
• Column Family: Name of the column family we wish to have in this instance.
• Age Rule: GC Age rule associated with the column family in the instance.
• Version Rule: GC version rule associated with the column family in the instance.
• Apply Type: Do we want to take an intersection/union of the above rules.

Input Field

Output

Drop Table

Drop the table in a given instance. We need two inputs to drop a table.

• Instance Name: Name of your Google BigTable Instance.

• Table Name: Name of the table we wish to drop in the above instance.

Input Field

Output

Update Permissions

Update the permission (Service account ID: Role mapping) associated with the bigtable instance. We need the below inputs for that:

• Policy - Policy in JSON format. Example policy is: {"bindings": [{"role": "roles/bigtable.admin", "members": ["serviceAccount:[email protected]", "serviceAccount:[email protected]"]}, {"role": "roles/bigtable.viewer", "members": ["serviceAccount:[email protected]","serviceAccount:[email protected]"]}]}

Output

Release Notes

• v2.0.0 - Updated architecture to support IO via filesystem