ploosh.

Documentation

Databricks

This connector is used to query a Databricks SQL warehouse or cluster using SQL.

Connection configuration

NameMandatoryDefaultDescription
tokenyesPersonal Access Token (PAT) for authentication
hostnameyesDatabricks instance hostname (e.g. adb-1234567890.1.azuredatabricks.net)
databaseyesDatabase name
http_pathyesHTTP path for the SQL warehouse or cluster (found in JDBC/ODBC settings)
portno443Port number

Example

connections:
  databricks_connection:
    type: databricks
    token: $var.databricks_token
    hostname: adb-1234567890.1.azuredatabricks.net
    database: my_database
    http_path: /sql/1.0/warehouses/abc123

Test case configuration

NameMandatoryDefaultDescription
queryyesSQL query to execute

Example

Example Databricks:
  source:
    type: databricks
    connection: databricks_connection
    query: |
      SELECT *
      FROM employees
      WHERE hire_date < '2000-01-01'
  expected:
    type: csv
    path: data/employeesbefore2000.csv

Requirements

  • pip install databricks-sqlalchemy (included in ploosh full installation)