GreaterThan, LLC

A lightweight, event-driven, pipeline framework.

Installation

From the command line:

pip install pipelayer

Getting Started

Download this simple application and follow the instructions in the README:

PipeLayer.getting_started.zip

Or, create the same application following these steps:

Step 1: Create Pipeline Filters

hello_world_filters.py

from pipelayer import Filter


class HelloFilter(Filter):
    def run(self, data, context):
        return "Hello"


class WorldFilter(Filter):
    def run(self, data, context):
        return f"{data},  World!"

functions.py

def create_message(data, context):
    return {"message": data}

Step 2: Create a Pipeline

Create a module to run the pipeline:

app.py

import json
from pipelayer import Pipeline
from pipelayer.util import render_manifest

from functions import create_message
from hello_world_filters import HelloFilter, WorldFilter


if __name__ == "__main__":
    hello_world_pipeline = Pipeline(
        [
            HelloFilter,
            WorldFilter,
            create_message,
            lambda data, context: json.dumps(data),
        ]
    )

    output = hello_world_pipeline.run(None)

    # output = '{"message": "Hello, World!"}'

    print("\nPipeline Output:")
    print(output)
    print("\nManifest:")
    print(render_manifest(hello_world_pipeline.manifest))

Step 3: Run the Pipeline

from the command line:

run app.py