Why we created Chaos Sumo – Smart Object Storage for Amazon S3

Internet-based computing is quickly moving towards a stateless model of operations that will only accelerate as pervasive online and mobile cloud applications take hold. This in turn is driving demand for Object Storage which is a much better fit for stateless operations than traditional storage. Object Storage is quickly becoming the cornerstone of cloud computing and the bedrock foundation for today’s mobile, web, and cloud services. Services like Amazon S3, the market leader, are growing at an amazing clip. Object storage is cheap, reliable and incredibly scalable and eliminates a historically big challenge in bringing new and compelling products to market. But S3, and similar services, are schemaless and lack the tools to extract value from the massive amounts of data that they house. Companies are pouring so much data into the cloud but then have to move it to other services to draw insights and perform even basic analytics. That’s expensive, time consuming and in many cases complete overkill.

We live in the pack rat age of data growth – no one deletes data. With more regulations demanding data retention, more IoT devices creating and capturing data, and more appetite for information and services, capacity and management challenges have become acute. That’s why we created our Smart Object Storage service, Chaossumo.io, to dramatically extend the value of cloud object storage, starting with S3.

Chaossumo.io is an abstraction layer that sits on top of S3. It’s an elastic, always-on service that dramatically extends the functionality of S3 and allows users easily discover, refine and query their data right where it lives within S3!  Think of it as a high definition lens or search tool into S3 data.  Users can quickly get more out of their cloud object storage without moving files, replicating storage or deploying expensive analytics scaffolding. Chaossumo.io turbocharges S3 adding powerful new API extensions through a flexible and intuitive drag and drop console.

Instant and insightful discovery Quickly discover what’s in your S3 buckets – overall file distribution, the structure of your buckets, the size, type, and history of your files. Our discover functionality lets you look deep inside files including csv, log, text and other file types right where they live.

Integrated Refine and Query – Turns S3 buckets into query-able result sets withcustomizable filters called “object groups” which can be refined into result sets called “virtual buckets”.  “Virtual buckets” can be queried or published to higher-level services for further analytics.

Intelligent Data Platform – A programmable S3-compatible API with discover, refine and query extensions. Data manipulation is in-place and virtual, requiring no physical transformation or external movement. Code directly on S3; build virtual data pipelines, transforms, and triggers. Quickly and easily integrate with higher-level services like Amazon Athena, Redshift, QuickSight, Snowflake, Tableau or Looker.

Visual Data Studio – One click auto discovery and cataloging. Drag and drop organization and data pipelines. Search engine for quick deep dive data analysis.

We are a team of data geeks and developers who have all come to rely on cloud services, particularly storage. We’ve all built applications and services that leverage the power of the cloud but became frustrated with the lack of tools to draw insights and value from our data. In order to answer even the most basic questions we found ourselves having to stand up expensive analytics scaffolding and move our data into other services which is always a complex and time consuming process. In some instances we had to hire entire teams whose responsibility was to deploy and manage analytics infrastructure. The bottom line is it simply took too long to answer questions, draw insights from and build on top of the data we were storing in the cloud.

Our mission is to eliminate that chaos and turn cloud object storage into an intelligent data platform. With chaossumo.io’s powerful discovery capabilities, visual data refinement studio, and RESTful API, organizations can quickly get more out of their cloud object storage. Users can browse, view and query their data right where it lives – on S3. That is Smart Object Storage.

Invitations to the Chaos Sumo beta program can be requested at Chaossumo.io.

What’s in your bucket?