Open-source visual ETL
Build it on the canvas. Work with it in Python. Open it as a graph again. Flowfile keeps your data work connected, on your own machine.
Free and MIT licensed. Desktop, Python, or Docker.
Download the desktop app, or install with Python. The editor, catalog, SQL tools and scheduler come together.
Flowfileon your machinepip install flowfile
flowfile run uiOne install gives you a place to build and run your data work.
Open an Excel file, read from a database, or connect to cloud storage and APIs. Save connections to use them in your next flow.
Select a source to see its connectors.
Your flowRead from the source.Join, filter, pivot, fuzzy match, or write a formula. See the columns and preview the data at each step. Add Python when you need it.
Join
Group by
CatalogResult: sales by region Example data
| region str | revenue f64 |
|---|---|
| North | 400.00 |
| South | 85.00 |
A repeatable pipeline you can inspect, explain and run again.
Store tables in the catalog, analyse them with SQL or Python, and create dashboards. Publish registered flows as endpoints for other applications.
Your data, analysis and the flows behind them stay connected.
Run on a schedule or when a catalog table changes. Check run history, see which nodes ran, and configure notifications when something fails.
Run history · Node status · Failure notifications
The flow refreshes the table your dashboard reads. The host machine needs to be running.
Write custom nodes in Python, install community nodes, or use isolated kernels with your own packages. An optional assistant can help build the graph.
def clean_names(df):
return df.with_columns(
pl.col("name")
.str.strip_chars()
.str.to_lowercase()
)
Clean namesYour own nodePackage your rules as a reusable node, with settings and a data preview.
Reuse your own transformations alongside the built-in nodes.
The visual editor, the catalog, and the tools around them.
Connect nodes to turn raw data into something useful. Inspect the columns before a run and preview the rows at each step.
Organise tables and flows in catalogs and schemas. The catalog connects each result to the work that produced it.
Query catalog tables in SQL, or work in a Python notebook with your own packages. Save useful analysis next to the data.
Create charts over catalog data and put them together in a dashboard. Use shared filters to explore the numbers.
Publish a registered flow as an HTTP endpoint from the catalog. Define its parameters and give each consumer an API key.
Set an interval, a cron schedule, or a trigger on catalog table updates. Each run is recorded so you can see what happened.
Put your own Python logic in a reusable node. Define its settings, test it in the Node Designer, and use it alongside the built-in nodes.
WHY FLOWFILE
The canvas and the Python API describe the same graph. Choose the view that makes the work easier to build, check, and share.
How code export worksBuild on the canvas or with the FlowFrame API. Export supported file and transform pipelines as standalone Polars scripts.
Joins, filters, and aggregations run on Polars. Inspect column types and preview intermediate results while you build.
Run locally or self-host. Save flows as readable YAML that you can keep in version control. Flowfile is free and MIT licensed.
On the canvas
In code
import flowfile as fffrom flowfile import col orders = ff.read_csv("orders.csv")customers = ff.read_parquet("customers.parquet") sales = ( orders.join(customers, on="customer_id") .filter(col("amount") > 100) .group_by("region") .agg(col("amount").sum().alias("revenue")))sales.write_parquet("sales_by_region.parquet")import polars as pl def run_etl_pipeline(): orders = pl.scan_csv("orders.csv") customers = pl.scan_parquet("customers.parquet") sales = ( orders.join(customers, on="customer_id") .filter(pl.col("amount") > 100) .group_by("region") .agg(pl.col("amount").sum().alias("revenue")) ) sales.sink_parquet("sales_by_region.parquet")One graph, two views. Hover a node to see its code and its columns, or switchSwitch to the plain Polars script Flowfile exports for a flow like this.
YOUR FIRST FLOW
Open the sample sales flow in your browser. Select a node, inspect the data, and change a step to see what happens.
Open the browser demoRuns in your browser. Best explored on a desktop.
Install Flowfile to connect databases and work with larger datasets.
A sales pipeline is already on the canvas.
Select a node to see what each operation does.
Install Flowfile, add a file, and build your first flow.
Choose the desktop app for your own machine, Python for a package install, or Docker for a shared setup.
Have a question or something you’d like to build with Flowfile? Let’s talk. You can also sponsor the project to help fund its development.