GreaterThan, LLC

After two years of working on a few microservices projects (mostly Python-based), with varying architectures and program flow, and after lengthy brainstorming discussions with one of the client engineers on my current project, I started to come up with a rough idea of a generic, reusable framework. Within a week after writing the first line of code, I had a working prototype, and quickly* published pipelayer 0.1.0, a lightweight pipeline framework based on the pipes/filters pattern.

Here’s a sample:

from app_context import AppContext 
from app_settings import AppSettings 
from pipelayer import Pipeline 
from hello_filter import HelloFilter 
from world_filter import WorldFilter 
from logging import getLogger


app_settings = AppSettings()
app_context = AppContext(
    app_settings, 
    getLogger(__name__)
)
pipeline = Pipeline.create(
    app_context, 
    "Hello World Pipeline"
) 
output = pipeline.run([
     HelloFilter(),
     WorldFilter()
])
print(f"Pipeline Output: {output}") print(pipeline.manifest.__dict__)

More documentation for this version is available on PyPi.

* quickly (adv): A relative term as it applies to first-time development of a Python package: hours of research, countless executions of setup.py, more research, a few runs of installing the package from the local wheel, and finally, success!