dremio_client package

Submodules

dremio_client.cli module

Console script for dremio_client.

dremio_client.cli.print_version(ctx, param, value)[source]

dremio_client.dremio_client module

Main module.

class dremio_client.dremio_client.DremioClient(config)[source]

Bases: object

data
get_item(cid=None, path=None)[source]
group(gid=None, name=None)[source]

return details for a group

User must supply one of gid or name. gid takes precedence if both are supplied .. note:: can only be run by admin

Parameters:
  • gid – group id
  • name – group name
Raise:

DremioUnauthorizedException if token is incorrect or invalid
Raise:

DremioPermissionException user does not have permission
Raise:

DremioNotFoundException group could not be found
Returns:

group info as a dict
personal_access_token(uid)[source]

return a list of personal access tokens for a user

Note

can only be run for the logged in user

Parameters:uid – user id
Raise:DremioUnauthorizedException if token is incorrect or invalid
Raise:DremioPermissionException user does not have permission
Raise:DremioNotFoundException user could not be found
Returns:personal access token list
query(sql, pandas=True, method='flight')[source]
reflections
simple()[source]
user(uid=None, name=None)[source]

return details for a user

User must supply one of uid or name. uid takes precedence if both are supplied .. note:: can only be run by admin

Parameters:
  • uid – group id
  • name – group name
Raise:

DremioUnauthorizedException if token is incorrect or invalid
Raise:

DremioPermissionException user does not have permission
Raise:

DremioNotFoundException user could not be found
Returns:

user info as a dict
votes
wlm_queues
wlm_rules

dremio_client.dremio_simple_client module

class dremio_client.dremio_simple_client.SimpleClient(config)[source]

Bases: object

cancel_job(jobid)[source]

cancel running job with job id = jobid

https://docs.dremio.com/rest-api/jobs/post-job.html

Parameters:jobid – id of the job to cancel
Returns:result object

:exception DremioNotFoundException no job found :exception DremioBadRequestException job already finished

catalog()[source]
catalog_item(cid, path)[source]
collaboration_tag(cid)[source]

returns a list of tags for catalog entity

Parameters:cid – catalog entity id
Raise:DremioBadRequestException if tags can’t exist on this entity
Raise:DremioUnauthorizedException if token is incorrect or invalid
Raise:DremioPermissionException user does not have permission
Raise:DremioNotFoundException user could not be found
Returns:list of tags
collaboration_wiki(cid)[source]

returns a wiki details for catalog entity

Parameters:cid – catalog entity id
Raise:DremioUnauthorizedException if token is incorrect or invalid
Raise:DremioPermissionException user does not have permission
Raise:DremioNotFoundException user could not be found
Returns:wiki details
create_queue(json)[source]

create a single queue

https://docs.dremio.com/rest-api/reflections/post-wlm-queue.html

Parameters:json – json document for new queue
Returns:result object
create_reflection(json)[source]

create a single reflection

https://docs.dremio.com/rest-api/reflections/post-reflection.html

Parameters:json – json document for new reflection
Returns:result object
delete_catalog(cid, tag)[source]

remove a catalog item from Dremio

https://docs.dremio.com/rest-api/catalog/delete-catalog-id.html

Parameters:
  • cid – id of a catalog entity
  • tag – version tag of entity
Returns:

None
delete_personal_access_token(uid)[source]

create a pat for a given user

https://docs.dremio.com/rest-api/user/delete-user-uid-token.html

Parameters:uid – id user
Returns:updated catalog entity
delete_queue(queueid)[source]

delete a single queue by id

https://docs.dremio.com/rest-api/reflections/delete-wlm-queue.html

Parameters:queueid – id of the queue to delete
Returns:result object
delete_reflection(reflectionid)[source]

delete a single reflection by id

https://docs.dremio.com/rest-api/reflections/delete-reflection.html

Parameters:reflectionid – id of the reflection to fetch
Returns:result object
graph(cid)[source]
group(gid=None, name=None)[source]

return details for a group

User must supply one of gid or name. gid takes precedence if both are supplied .. note:: can only be run by admin

Parameters:
  • gid – group id
  • name – group name
Raise:

DremioUnauthorizedException if token is incorrect or invalid
Raise:

DremioPermissionException user does not have permission
Raise:

DremioNotFoundException group could not be found
Returns:

group info as a dict
job_results(jobid)[source]
job_status(jobid)[source]
modify_queue(queueid, json)[source]

update a single queue by id

https://docs.dremio.com/rest-api/reflections/put-wlm-queue.html

Parameters:
  • queueid – id of the reflection to fetch
  • json – json document for modified queue
Returns:

result object
modify_reflection(reflectionid, json)[source]

update a single reflection by id

https://docs.dremio.com/rest-api/reflections/put-reflection.html

Parameters:
  • reflectionid – id of the reflection to fetch
  • json – json document for modified reflection
Returns:

result object
modify_rules(json)[source]

update wlm rules. Order of rules array is important!

The order of the rules is the order in which they will be applied. If a rule isn’t included it will be deleted new ones will be created https://docs.dremio.com/rest-api/reflections/put-wlm-rule.html

Parameters:json – json document for modified reflection
Returns:result object
personal_access_token(uid)[source]

return a list of personal access tokens for a user

Note

can only be run for the logged in user

Parameters:uid – user id
Raise:DremioUnauthorizedException if token is incorrect or invalid
Raise:DremioPermissionException user does not have permission
Raise:DremioNotFoundException user could not be found
Returns:personal access token list
promote_catalog(cid, json)[source]

promote a catalog entity

https://docs.dremio.com/rest-api/catalog/put-catalog-id.html

Parameters:
  • cid – id of catalog entity
  • json – json document for new catalog entity
Returns:

updated catalog entity
query(query, context=None, sleep_time=10, asynchronous=False)[source]

Run a single sql query asynchronously

This executes a single sql query against the rest api asynchronously and returns a future for the result

Parameters:
  • query – valid sql query
  • context – optional context in which to execute the query
  • sleep_time – seconds to sleep between checking for finished state
  • asynchronous – boolean execute asynchronously
Raise:

DremioException if job failed
Raise:

DremioUnauthorizedException if token is incorrect or invalid
Returns:

concurrent.futures.Future for the result
Example: 
>>> client =
>>> f = query('select * from sys.options', asynchronous=True)
>>> f.result()
[{'record':'1'}, {'record':'2'}]
reflection(reflectionid)[source]
reflections(summary=False)[source]
refresh_metadata(table)[source]

Refresh the metadata for a given physical dataset

Parameters:table – the physical dataset to be refreshed
Raise:DremioException if job failed
Raise:DremioUnauthorizedException if token is incorrect or invalid
Returns:None
refresh_pds(pid)[source]

refresh a physical dataset and all its child reflections

https://docs.dremio.com/rest-api/catalog/post-catalog-id-refresh.html

Parameters:pid – id of a catalog entity
Returns:None
refresh_reflections_of_one_dataset(path)[source]

Refresh the reflection for a given dataset Refresh will disable and enable reflection. As long reflection will be reenabled all queries will be redirected to source dataset. All VDS Reflection derived from this VDS will be refreshed as well

Parameters:path – list [‘space’, ‘folder’, ‘vds’]
Returns:None
refresh_vds_reflection_by_path(path)[source]

Refresh the reflection for a given virtual dataset

Parameters:path – list [‘space’, ‘folder’, ‘vds’]
Returns:None
set_catalog(json)[source]

add a new catalog entity

https://docs.dremio.com/rest-api/catalog/post-catalog.html

Parameters:json – json document for new catalog entity
Returns:new catalog entity
set_collaboration_tag(cid, tags)[source]

returns a list of tags for catalog entity

Parameters:
  • cid – catalog entity id
  • tags – string list
Raise:

DremioBadRequestException if tags can’t exist on this entity
Raise:

DremioUnauthorizedException if token is incorrect or invalid
Raise:

DremioPermissionException user does not have permission
Raise:

DremioNotFoundException user could not be found
Returns:

list of tags
set_personal_access_token(uid, label, lifetime=24)[source]

create a pat for a given user

https://docs.dremio.com/rest-api/user/post-user-uid-token.html

Parameters:
  • uid – id user
  • label – label of token
  • lifetime – lifetime in hours of token
Returns:

updated catalog entity
sql(query, context=None)[source]
update_catalog(cid, json)[source]

update a catalog entity

https://docs.dremio.com/rest-api/catalog/put-catalog-id.html

Parameters:
  • cid – id of catalog entity
  • json – json document for new catalog entity
Returns:

updated catalog entity
user(uid=None, name=None)[source]

return details for a user

User must supply one of uid or name. uid takes precedence if both are supplied .. note:: can only be run by admin

Parameters:
  • uid – group id
  • name – group name
Raise:

DremioUnauthorizedException if token is incorrect or invalid
Raise:

DremioPermissionException user does not have permission
Raise:

DremioNotFoundException user could not be found
Returns:

user info as a dict
votes()[source]

return details all reflection votes

Summarizes all votes in the system https://docs.dremio.com/rest-api/votes/get-vote.html .. note:: can only be run by admin

Raise:DremioUnauthorizedException if token is incorrect or invalid
Raise:DremioPermissionException user does not have permission
Returns:votes as a list of dicts
wlm_queues()[source]

return details all workload management queues

Summarizes all workload management queues in the system https://docs.dremio.com/rest-api/wlm/get-wlm-queue.html .. note:: can only be run by admin .. note:: Enterprise only

Raise:DremioUnauthorizedException if token is incorrect or invalid
Raise:DremioPermissionException user does not have permission
Raise:DremioNotFoundException queues not found
Returns:queues as a list of dicts
wlm_rules()[source]

return details all workload management rules

Summarizes all workload management rules in the system https://docs.dremio.com/rest-api/wlm/get-wlm-rule.html .. note:: can only be run by admin .. note:: Enterprise only

Raise:DremioUnauthorizedException if token is incorrect or invalid
Raise:DremioPermissionException user does not have permission
Raise:DremioNotFoundException ruleset is not found
Returns:rules as a list of dicts

dremio_client.error module

exception dremio_client.error.DremioBadRequestException(msg, original_exception, response=None)[source]

Bases: dremio_client.error.DremioException

exception dremio_client.error.DremioException(msg, original_exception, response=None)[source]

Bases: Exception

base dremio exception

exception dremio_client.error.DremioNotFoundException(msg, original_exception, response=None)[source]

Bases: dremio_client.error.DremioException

exception dremio_client.error.DremioPermissionException(msg, original_exception, response=None)[source]

Bases: dremio_client.error.DremioException

exception dremio_client.error.DremioUnauthorizedException(msg, original_exception, response=None)[source]

Bases: dremio_client.error.DremioException

Module contents

dremio_client.init(config_dir=None, simple_client=False, config_dict=None)[source]

create a new Dremio client object

This returns a rich client by default. This client abstracts the Dremio catalog into a a set of Python objects and enables <Tab> completion where possible. It also supports fetching datasets via flight or odbc

The simple client simply wraps the Dremio REST endpoints an returns dict objects

Parameters:
  • config_dir – optional directory to look for config in
  • simple_client – return the ‘simple’ client.
  • config_dict – dictionary of extra config arguments
Returns:

either a simple or rich client
Example: 
>>> client = init('/my/config/dir')
dremio_client.catalog(token, base_url, ssl_verify=True)[source]

https://docs.dremio.com/rest-api/catalog/get-catalog.html populate the root dremio catalog

Parameters:
  • token – auth token from previous login attempt
  • base_url – base Dremio url
  • ssl_verify – ignore ssl errors if False
Returns:

json of root resource
dremio_client.catalog_item(token, base_url, cid=None, path=None, ssl_verify=True)[source]

fetch a specific catalog item by id or by path

https://docs.dremio.com/rest-api/catalog/get-catalog-id.html https://docs.dremio.com/rest-api/catalog/get-catalog-path.html

Parameters:
  • token – auth token from previous login attempt
  • base_url – base Dremio url
  • cid – unique dremio id for resource
  • path – list [‘space’, ‘folder’, ‘vds’]
  • ssl_verify – ignore ssl errors if False
Returns:

json of resource
dremio_client.sql(token, base_url, query, context=None, ssl_verify=True)[source]

submit job w/ given sql

https://docs.dremio.com/rest-api/sql/post-sql.html

Parameters:
  • token – auth token
  • base_url – base Dremio url
  • query – sql query
  • context – optional dremio context
  • ssl_verify – ignore ssl errors if False
Returns:

job id json object
dremio_client.job_status(token, base_url, job_id, ssl_verify=True)[source]

fetch job status

https://docs.dremio.com/rest-api/jobs/get-job.html

Parameters:
  • token – auth token
  • base_url – sql query
  • job_id – job id (as returned by sql)
  • ssl_verify – ignore ssl errors if False
Returns:

status object
dremio_client.job_results(token, base_url, job_id, offset=0, limit=100, ssl_verify=True)[source]

fetch job results

https://docs.dremio.com/rest-api/jobs/get-job.html

Parameters:
  • token – auth token
  • base_url – sql query
  • job_id – job id (as returned by sql)
  • offset – offset of result set to return
  • limit – number of results to return (max 500)
  • ssl_verify – ignore ssl errors if False
Returns:

result object