Connecting to A.S.K with Jupyter

Connect jupyter to kafka

‍

A step-by-step guide to integrating Jupyter with Streambased, unlocking powerful capabilities for interactive data exploration and analysis on streaming data.

‍

Pre-requisites

‍

Install the following packages in your python environment

‍

pip install jupyterlab
pip install jupysql
pip install sqlalchemy-trino 
pip install pandas

‍

Step 1: Start the notebook

‍

Launch a notebook directly with:

‍

jupyter lab

‍

Step 2: Create Database Engine

‍

From your notebook create a database engine using sqlalchemy.engine

‍

from sqlalchemy.engine import create_engine
engine = create_engine("trino://streambased.cloud:8443/kafka",
                       connect_args ={"http_scheme":"https", "schema":"streambased"})

‍

‍

Step 3: Load the SQL extension

‍

From your notebook load the SQL extension:

‍

%load_ext sql

‍

Step 4: Connect SQL engine to Database

‍

From your notebook connect sql engine to database:

‍

%sql engine

‍

Step 5: Run a query

‍

Now we can run a query:

‍

%sql SELECT * FROM demo_transactions

‍

Step 6: (optional) Pandas?

‍

Change the query to pandas dataframe

‍

transactions = %sql SELECT * FROM demo_transactions
df = result.DataFrame()

‍

‍

‍