Internal site. Jolli authentication required to view.
Skip to Content
🚀 Getting StartedFirst Pipeline

Last Updated: 3/19/2026

Building Your First Pipeline

This guide walks you through creating a complete data pipeline in Feldera. You’ll define SQL tables and views, insert data, observe incremental updates, and learn how Feldera’s incremental computation works in practice.

The Use Case

We’ll build a pipeline that ingests data about vendors, parts, and prices, and continuously tracks the lowest available price for each part across all vendors. This is a common pattern in supply chain analytics and price monitoring systems.

Prerequisites

Make sure you have Feldera running locally by following the Quick Start guide. Open the Feldera Web Console at http://localhost:8080.

Step 1: Create the Pipeline

In the Feldera Web Console, create a new pipeline named “supply_chain” and paste the following SQL code in the editor:

CREATE TABLE VENDOR (
    id BIGINT NOT NULL PRIMARY KEY,
    name VARCHAR,
    address VARCHAR
) WITH ('materialized' = 'true');
 
CREATE TABLE PART (
    id BIGINT NOT NULL PRIMARY KEY,
    name VARCHAR
) WITH ('materialized' = 'true');
 
CREATE TABLE PRICE (
    part BIGINT NOT NULL,
    vendor BIGINT NOT NULL,
    price INTEGER
) WITH ('materialized' = 'true');
 
-- Lowest available price for each part across all vendors.
CREATE VIEW LOW_PRICE (
    part,
    price
) AS
    SELECT part, MIN(price) AS price FROM PRICE GROUP BY part;
 
-- Lowest available price for each part along with part and vendor details.
CREATE MATERIALIZED VIEW PREFERRED_VENDOR (
    part_id,
    part_name,
    vendor_id,
    vendor_name,
    price
) AS
    SELECT
        PART.id AS part_id,
        PART.name AS part_name,
        VENDOR.id AS vendor_id,
        VENDOR.name AS vendor_name,
        PRICE.price
    FROM
        PRICE,
        PART,
        VENDOR,
        LOW_PRICE
    WHERE
        PRICE.price = LOW_PRICE.price AND
        PRICE.part = LOW_PRICE.part AND
        PART.id = PRICE.part AND
        VENDOR.id = PRICE.vendor;

Understanding the SQL

The first part declares three input tables using CREATE TABLE statements. These define the schema for your input data. Note the 'materialized' = 'true' attribute—this instructs Feldera to store the entire contents of the table so you can browse it at any time.

The second part defines queries using SQL views. The LOW_PRICE view computes the minimum price for each part across all vendors. The PREFERRED_VENDOR view builds on LOW_PRICE to show which vendor offers the best price for each part, along with part and vendor details.

We declare PREFERRED_VENDOR as a materialized view, which means Feldera stores its entire contents for browsing. Regular views (like LOW_PRICE) only allow you to observe a stream of changes, not query the current state.

Click the Play button (▶) to start the pipeline.

Step 2: Insert Data

When the pipeline is running, it can process incoming changes. Let’s populate the tables using ad-hoc queries.

Open the “Ad-hoc query” tab and insert vendor data:

INSERT INTO VENDOR (id, name, address) VALUES
(1, 'Gravitech Dynamics', '222 Graviton Lane'),
(2, 'HyperDrive Innovations', '456 Warp Way'),
(3, 'DarkMatter Devices', '333 Singularity Street');

Press Enter or click the Play button to execute. You should see a result showing 3 rows inserted.

Verify the data was inserted:

SELECT * FROM VENDOR;

Now insert parts and prices:

INSERT INTO PART (id, name) VALUES
(1, 'Flux Capacitor'),
(2, 'Warp Core'),
(3, 'Kyber Crystal');
INSERT INTO PRICE (part, vendor, price) VALUES
(1, 2, 10000),
(2, 1, 15000),
(3, 3, 9000);

Step 3: Query the Results

Feldera is an incremental view maintenance (IVM) engine. When new data arrives in tables, views are automatically updated. Feldera computes only what has changed, with resource consumption proportional to the size of the changes, not the entire dataset.

When you inserted data into the tables, Feldera already computed the results for the views. Let’s inspect the PREFERRED_VENDOR view:

SELECT part_name, vendor_name FROM PREFERRED_VENDOR;

You should see which vendor offers the best price for each part:

  • Flux Capacitor → HyperDrive Innovations
  • Warp Core → Gravitech Dynamics
  • Kyber Crystal → DarkMatter Devices

Note that you can only use ad-hoc SELECT queries to inspect materialized tables and views. The LOW_PRICE view is not declared as MATERIALIZED, so you cannot query it directly. However, SQL views declared in your main program can reference both materialized and non-materialized tables and views.

Step 4: Observe Incremental Updates

Let’s see Feldera’s incremental computation in action. Instead of querying views, we’ll capture changes as they happen.

Open the “Change Stream” tab and check the boxes next to the PRICE table, LOW_PRICE view, and PREFERRED_VENDOR view.

Switch back to the “Ad-hoc query” tab and execute:

INSERT INTO PRICE (part, vendor, price) VALUES
(2, 3, 12000);

This introduces a new, lower price for the Warp Core from a different vendor.

Navigate to the “Change Stream” tab and observe the updates. You’ll see:

  • An insert in the PRICE table
  • Inserts and deletes for the views

Feldera incrementally updates views by deleting old records that are no longer part of the view and inserting newly added records. This is much more efficient than recomputing the entire view from scratch.

Query the PREFERRED_VENDOR view again:

SELECT part_name, vendor_name FROM PREFERRED_VENDOR;

The entry for “Warp Core” has been updated to show “DarkMatter Devices” as the preferred vendor, reflecting the new lower price.

Step 5: Clean Up

When you’re done, shut down the pipeline by clicking the Stop button (■). This clears all ingested data, computed view results, and any accumulated internal state.

Key Takeaways

Let’s recap what you’ve learned:

  • Feldera executes programs written in standard SQL, using CREATE TABLE and CREATE VIEW statements

    • CREATE TABLE defines schemas for input data
    • CREATE VIEW defines queries over tables and other views

  • An SQL program is instantiated as part of a pipeline

  • Feldera evaluates SQL programs incrementally, continuously updating results as input data changes

  • You can observe changes as they happen or query snapshots of materialized tables and views

  • When a pipeline stops, all processed inputs and computed results are lost

What’s Next

  • Pipelines Core Concepts: Dive deeper into the fundamental building blocks of Feldera—pipelines, tables, views, and the pipeline lifecycle
  • Connectors Overview: Learn how to connect pipelines to external data sources like Kafka, HTTP endpoints, and data lakes
Cloud Storage ConnectorsQuick Start