MIME type which is used for Google Spreadsheets when converted to structured records. MIME type which is used for Google Documents when converted to structured records. “Bytes” and “String” values are available. Is not applicable for files in Google formats. Maximum body size for each structured record specified in bytes. Service Account JSON can be generated on Google Cloud Service Account page. Contents of the service account JSON file. Otherwise, the preview as well as pipeline run will fail with insufficient permission error.

When set to auto-detect, the GCE VM needs to be configured with the scope required by the Google Drive API. When running on other clusters, the file must be present on every node in the cluster. Can be set to auto-detect when running on a Dataproc cluster. Path on the local file system of the service account key used for authorization. Viewer role must be granted to the specified service account to read files from the Google Drive Folder. Make sure that the Google Drive Folder is shared with the specified service account email. OAuth2 Refresh Token to acquire new access tokens. OAuth2 Client Secret used to access the authorization server. OAuth2 Client ID used to identify the application. OAuth2 client credentials can be generated on Google Cloud Credentials Page.įor more information about OAuth2, see Google Drive API Documentation. Make sure Google Drive API is enabled in the GCP Project.

OAuth2 and Service Account types are available. Type of authentication used to access Google API. For Google Drive formats user should specify exporting format in Exporting section. The following values are supported: binary (all non-Google Drive formats), Google Documents, Google Spreadsheets, Google Drawings, Google Presentations and Google Apps Scripts. Types of files which should be pulled from a specified directory. For Custom selection the dates range can be specified via Start Date and End Date. User can select either among predefined or custom entered ranges. Filter that narrows set of files by modified date range. Filters follow the Google Drive filters syntax. Filter that can be applied to the files in the selected directory. Descriptions for properties can be view at Drive API file reference. They will be a part of output structured record. Properties that represent metadata of files. Then the Directory Identifier would be 1dyUEebJaFnWa3Z4n0BFMVAXQ7mfUH11g.

Google drive api documentation