Welcome to the eKuiper Wiki

LOGOImage Added

Overview

LF Edge eKuiper is

a lightweight IoT data analytics

and stream processing engine running on resource-constraint edge devices.

The major goal

for eKuiper is to

provide a streaming software

Image Removed

framework (similar to Apache Flink) in edge side. eKuiper's rule engine allows user to provide either SQL based or graph based (similar to Node-RED) rules to create IoT edge analytics applications within few minutes.

Image Added

User scenarios

It can be run at various IoT edge

user scenarios, such as

,

• Real-time processing of production line data in the IIoT

• Gateway of

• connected vehicle analyze the data from

• CAN in IoV
• Real-time analysis of

• wind turbines and smart bulk energy storage data in smart

• energy

eKuiper processing at the edge can greatly reduce system response latency, save network bandwidth and storage costs

and improve system security.

Features

• Lightweight

  • Core server package is only about 4.5M,

• • memory footprint is about 10MB

• Cross-platform

  • CPU Arch：X86 AMD * 32

• • /64; ARM * 32

• • /64; PPC

• • Popular Linux distributions, OpenWrt Linux, MacOS and Docker
  • Industrial PC, Raspberry Pi, industrial gateway, home gateway, MEC edge cloud server

• Data analysis support

  • Support data

• • ETL
  • Data order, group, aggregation and join with different data sources (the data from databases and files)
  • 60+ functions, includes mathematical, string, aggregate and hash etc
  • 4 time windows & count window

• Highly

• extensible

  It supports to extend at Source, 

• Functions and Sink with Golang or Python.

  • Source:

• • allows users to add more data source for analytics.
  • Sink: allows users to send analysis result to different customized systems.
  • UDF functions: allow users to add customized functions for data analysis (for example, AI/ML function invocation)

• Management

  • A free

Management

• • web based management dashboard for

• • visualized management
  • Plugins, streams and rules management through CLI

• • , REST API and config maps(Kubernetes)
  • Easily be

• • integrated with Kubernetes framworks KubeEdge, OpenYurt, K3s 

• • Baetyl

• Integration with

• EMQX products

  Seamless integration with

•  EMQX, Neuron & NanoMQ, and provided an end-to-end solution from

• IIoT, IoV

Quick Start

• eKuiper 5 minutes quick start
• EdgeX rule engine tutorial

Performance Test Result

MQTT throughput test

• Using JMeter MQTT plugin to send simulation data to EMQ X Broker, such as: {"temperature": 10, "humidity" : 90}, the value of temperature and humidity are random integer between 0 - 100.
• eKuiper subscribe from EMQ X Broker, and analyze data with SQL: SELECT * FROM demo WHERE temperature > 50
• The analysis result are wrote to local file by using file sink plugin.

EdgeX throughput test

• A Go application is wrote to send data to ZeroMQ message bus, the data is as following.

  {
    "Device": "demo", "Created": 000, …
    "readings": 
    [
       {"Name": "Temperature", value: "30", "Created":123 …},
       {"Name": "Humidity", value: "20", "Created":456 …}
    ]
  }
  

• eKuiper subscribe from EdgeX ZeroMQ message bus, and analyze data with SQL: SELECT * FROM demo WHERE temperature > 50. 90% of data will be filtered by the rule.

• The analysis result are sent to nop sink, all of the result data will be ignored.

Max number of rules support

• 8000 rules with 800 message/second
• Configurations
  • 2 core * 4GB memory in AWS
  • Ubuntu
• Resource usage
  • Memory: 89% ~ 72%
  • CPU: 25%
  • 400KB - 500KB / rule
• Rule
  • Source: MQTT
  • SQL: SELECT temperature FROM source WHERE temperature > 20 (90% data are filtered)
  • Sink: Log