Welcome to JobFlow

Struggling to figure out how to make use of the numerous cloud services available today? Often, determining how to best integrate the cloud into a system is not an easy task. Serverless offerings can be confusing and require a different architecture layout from traditional applications.

JobFlow is a .Net-based state-full, asynchronous job management application framework designed to promote the use of serverless cloud offerings, thereby allowing low-cost, highly scalable applications to be created in a shorter amount of time. JobFlow will take care of all routing, scheduling, and communication requirements, allowing developers to focus on writing the business logic. Use it to build a workflow or a state machine. Think of it as an orchestrator for your backend cloud entities - Lambdas or Functions, Containers, APIs, VMs, and others.

Features

• Configure AI rules to route work based on job or work item data
  • Or define your own processing engine (see IFlowEngine)
• Highly scalable
• Available automatic error recovery with retry scheduling
• Plugin Architecture
• Ability to schedule jobs for future execution - multiple scheduler implementations provided
  • Azure Service Bus Queues
  • Hangfire
• Multiple message transports - mix and match multiple different transports in a single system
  • Azure Storage Queues
  • Azure Service Bus Queues
  • Redis pub/sub
  • MQTT
  • Hangfire
• Multiple document storage systems - mix and match different storages based on the message type
  • Azure Storage
  • Redis
  • In-memory
  • On-disk files
• Keep cloud infrastructure costs low
• Cross the on-premise and cloud boundary to allow for partial cloud transitions
• System Monitoring UI
• Pre-built ASP.Net endpoints for common actions

Architecture

At its core, JobFlow is designed to integrate routing, persistance, error recovery, and all its other features into an application's code base rather than utilizing an external workflow product. As such, the system is used as an SDK where, other than environment-dependent options defined in the application's appSettings json file, all operations are done in code. This reduces the need for "band-aid" code required to integrate multiple different frameworks into a single application.

Architecture Description

The system can be run strictly as background operations within an ASP.Net or console application. Or the entire thing can be run as Functions on Azure, providing low-cost on-demand processing.

Getting Started

• Configuration - Example configuration
• Core Handlers - Core Handler configuration
• Function Deployment - Deploying JobFlow and workers as Azure Functions
• Message Classes - Message classes
• Rule API - Overview of Rule API additions to the NRule DSL
• Storage and Transport Plugins - Overview of Plugins

Examples

• Payment Processor - Gives simple step-by-step workflow example.
• Doorbell - Uses forward-chaining of rules to initiate jobs. Also gives a quick example of how to write a job handling Azure Function.
• Doorbell Part 2 - We make additions to the doorbell project to demonstrate some advanced features of the rules.
• Additional Examples - Additional example ideas.

Comparisons

• NServiceBus - Comparison to NServiceBus (or MassTransit).

Contact

Mark Henry
mark.henry@jobflow.biz

Copyright © 2021-2023 by Mark Henry