This connector is used to read Parquet files using Spark.

:warning: A spark connector can be use only with another spark connector. It is not possible to use a spark connector with a non spark connector.

See Spark documentation for more information.

Connection configuration

Configuration

Test case configuration

Example

Example Parquet Spark:
  source:
    type: parquet_spark
    path: data/employees/example.parquet
  expected:
    type: sql_spark
    query: |
      select * 
          from employees
          where hire_date < "2000-01-01"