Software developers at Lockheed Martin have designed a platform to make big data analysis easier for developers and non-developers and are open sourcing the project on GitHub, a popular web-based hosting service.
READ: Lockheed Recognized for Outstanding Service to Meteorology
The StreamFlow software project is designed to make working with Apache Storm, a free and open source distributed real-time computation system, easier and more productive.
A Storm application ingests significant amounts of data through the use of topologies, or set of rules that govern how a network is organized. These topologies categorize the data streams into understandable pipelines.
“The ultimate goal of StreamFlow is to make working with Storm easier and faster, allowing non-developers and domain experts of all kinds to contribute to real-time data-driven solutions,” said Jason O’Connor, vice president of Analysis & Mission Solutions with Lockheed Martin Information Systems & Global Solutions. “The next step in data analytics relies on the inclusion of diverse expertise and we envision this product contributing to fields ranging from systems telematics to cyber security to medical care.”
Companies currently using Apache Storm to repartition streams of data include Twitter, Spotify and The Weather Channel.
The StreamFlow software introduces a dashboard to monitor metrics, high-level protocols to make coding more interoperable, and a graphical topology builder to make assembling and monitoring topologies in Storm much easier for beginner programmers and users without software development experience.
StreamFlow was open sourced on Lockheed Martin’s GitHub account for users to install on their own desktops. Installing StreamFlow is much like installing typical web applications, though some configuration may be necessary. The software is released under the Apache 2 license and can be freely downloaded and built from GitHub.
The software contains a front end user interface supported by a series of web services. Future plans include open sourcing of additional frameworks and support for further real-time processing systems like Apache Spark.
Filed Under: Aerospace + defense