DataWeave

Wikipedia states: data-driven journalism as a journalistic process based on analyzing and filtering large data sets for the purpose of creating a news story. Data driven journalism enables reporters to tell meaningful stories with insights gleaned from data using various tools.

We are observing a growing trend in which Governments and other institutions are making their data publicly available. In addition, most of this data is available online. This presents a significant opportunity for the journalism industry.

Well, but the fly in the ointment is that this data is not readily consumable because

• The data is dispersed across various sources. Sifting through such dispersed data is a complex and time consuming task.
• The data is temporally changing. So, it is difficult to keep track of it.
• There is no standard representation for this data.

This means that once a potential data source is identified, there is the tedious task of cleaning, structuring and transforming the data to a consumable form. But this is self-defeating. Should a journalist really spend so much time and effort in collating data reports, while her actual goal is to create cohesive news stories? To put it in another way: one needs to be a technical person to derive value out of the huge amount of data available. However, typically it is a non-technical person such as a journalist or a market researcher who can make most sense and draw meaningful conclusions out of that data!

Imagine a situation where generating a data report is just a few clicks away. This is where DataWeave comes in. DataWeave makes the process of accessing data really easy, by aggregating data from various public sources and providing them via easy to access APIs in a consumable form. 

Using DataWeave’s Data APIs, a user can selectively query for data and use them for gaining insights. Our APIs can also be easily plugged into the user’s system – in this case, a journalist’s data report generation dashboard. Once the data is available in a standard, consumable format, it is easy to visualize or identify patterns in it using only a few lines of code, or by exporting it to spreadsheet tools.

Let’s consider a scenario in which a journalist wants to analyze the change in retail prices of commodities over time, for a particular city. She can use DataWeave’s Retail Commodities API to access this data. The required data can be queried through the dashboard provided in the link, for instance. Similarly, the API could be plugged into any existing report generation dashboard, for performing analysis.

Taking our own dashboard as an example, let us look at how the prices of potato have varied in Mumbai for the month of March, 2012.

You can then download the data as a csv file which can be imported to your favourite spreadsheet program or visualization kit. It is as easy as that!

A basic analysis of price changes can be visualized using MS Excel graphs, for instance.

Similarly, the API can be plugged into dashboards with rich visualizations and various “key performance indicator” analyses – the journalist’s dashboard, for instance!

Please share your thoughts and experiences with us at contact [AT] dataweave [DOT] in.

Slides from our presentation at the Open DataCamp held in Bangalore on March 24th, 2012. 

Adhering to our core philosophy, “make open data on the web easily accessible and consumable”, we have launched a few APIs around open data we have been monitoring for about an year now. How can these APIs help you? APIs reduce the development time of data-dependent apps considerably, as you can focus on more relevant aspects of your application, such as algorithm design, visualizations, application interfaces, etc, than be stuck in the data collection/curation phase.

We realized the power of this as we started consuming from our API firehose to build our first product - PriceWeave. Priceweave is built on top of the ecommerce product pricing API.

What’s so great about “yet another price comparison engine”, might be the first question in your mind. PriceWeave is more than just a price comparison engine. Our goal is to help businesses get access to data required to make informed decisions. These could range from a simple price monitoring service to real time product placement strategies, short term sales forecasting, deciding how best to spend investor dollars (other than bleeding on discounts).

PriceWeave is being used by our customers as a real time competitor monitoring engine. Priceweave can help businesses with:

1. Real time monitoring of prices across competitors.
2. Set alerts for price changes.
3. Easily integrate it with your own Inventory Management Systems to produce a sales forecast report.
4. Use the dashboard to mine through data easily. (By recently changed, By lowest priced competitor etc).

The screenshots here will give you a visual account of how PriceWeave operates.

Monitor your products in real time

Map your products to the products that we already have.

Surf through the data using a set of filters available on the dashboard.

Here is a list of most commonly asked questions about priceweave http://www.priceweave.com/help.php

PriceWeave is just one of the many possibilities that can be built by leveraging our ecommerce API. You can mash it up with the other APIs we provide and come up with interesting apps or visualizations. In fact, we would love to hear about such undertakings! Please share your thoughts and experiences with us @ contact[AT]dataweave[dot]in.

A month has passed by since the launch of DataWeave’s new website. We have a seen a steady increase in usage and a few paying customers.

DataWeave is about making data on the Web easily accessible and consumable. Be it in creating applications or gaining insights in data, the first step any user goes through is: “Get the right data in the right form”. Though a lot of data sources exist on the Web, it is hard to find these sources and access them in a standard format. We create data APIs that lets users easily access this data in standard formats. 

Some of the APIs that are available are eCommerce pricing data, Commodity prices, Wholesale price index. Check out the list of APIs available here.

We can think of a lot of use-cases that one can come up with using these APIs:

• An application to monitor commodity prices have any impact on the share prices of a company. Companies use raw materials and the prices of these raw materials have a certain impact on their sales and hence share prices.
• Impact of population growth on weather.
• Did the weather have any impact on the number of people who turned up to vote during elections?

We believe the use cases around this data are limitless.

The ability to create data mashups using these APIs and/or combine them with proprietary data opens up a lot of use cases around this data. For example, organizations are using the eCommerce pricing data API to monitor competition, educate vendors and a host of other applications.

Data journalists, market researchers, analysts and application developers will find this data immensely useful.

In the days to come, we plan to add more data APIs and add wrappers along with the documentation around this data to enable easy consumption of this data.

Would love to hear your feedback.

    DataWeave is a data marketplace that enables data-driven businesses to access, search and visualize vast repositories of curated data from the Web in a form they can derive value from.