Big Data Analytics : What’s Next?

While a majority of Fortune 1000 companies are en-route to understanding Hadoop and adopting it in their technology stack, startups in the bay area and elsewhere have started asking the important and inevitable question: “What’s Next?”. Hadoop for the first time has allowed us to analyze massive amounts of data without necessarily indulging in expensive proprietary hardware or software. However, adoption of Hadoop alone isn’t necessarily helping businesses make smarter decisions or unearth completely new facts that could lead to immense growth of top line. The power of scalable infrastructure needs to be supplemented with nifty data mining and machine learning tools, better visualization of results, and easier ways to track and analyze the findings over a period of time. Besides, there is the entire realm of real-time analytics, which is beyond the batch oriented nature of Hadoop.

The “Global Big Data Conference“, scheduled to take place at the Santa Clara Convention Center on January 28, 2013 answers some of these very important questions around what’s happening in the field of “big data” and what’s to come next. Its a great 1 day conference that has a lot of interesting topics, covered by an awesome line-up of great speakers. In order to take advantage of some favorable pricing please register by tomorrow (January 22, 2013) and save a whole $100, as compared to the onsite price. In addition, as a reader of my blog, don’t forget to take advantage of the additional 20% discount, which you can avail by using the code: “SHAS“. See you all there!

Geolocation in MongoDB at the Silicon Valley MongoDB User Group

Thanks to all of you, who were able to join me at the session last evening. Thanks much for the kind remarks some of you left behind on the meetup message board, post the session.  Its very rewarding to know that many of you enjoyed the session and found it very useful. I loved the many questions that were brought up and discussed in the room. Please feel free to send more questions by emailing them to me at st (at) treasuryofideas (dot) com. Alternatively, you could tweet them to me at @tshanky.

The presentation from last evening is available online at

For all those who are excited about MongoDB and would like to learn more please join me for “MongoDB in an Hour!” on February 1, 2013. The format of that session would be as follows:

  • 1 hour free video session, which will be made available online by or before Feb 1, 2013.
  • 4 half-hour Google+ Hangout sessions for live Q&A.
  • Unlimited number of Q&A opportunities over email (or over a forum if we create one)
  • (optional) 1 evaluation exam. Passing the exam would entitle you to a certificate of honor.

This 1 hour session is substantially subsidized and I only ask for $25 as suggested donation to cover some of the costs.

Research Papers and Videos on Google Bigtable, GFS, Chubby and MapReduce

Thanks all for coming to my talk today on sorted ordered column-family stores at the Silicon Valley Cloud Computing Meetup. Here are the links to the Google’s research papers and a couple of videos that relate to Bigtable (and Google App Engine internals):

Bigtable: A Distributed Storage System for Structured Data

The Chubby Lock Service for Loosely-Coupled Distributed Systems

The Google File System

MapReduce: Simplified Data Processing on Large Clusters

BigTable: A Distributed Structured Storage System —

Google I/O 2008 – App Engine Datastore Under the Covers —

Enjoy reading the research papers and watching the videos!

Getting Friendly With Document Databases

Here is a draft of what I plan to present in Part 1 of the NoSQL Series on Jan 24th at Fenwick & West in Mountain View, CA (The series has 4 parts in all. It runs between 1/24 and 1/27, everyday at 7pm). The event is  hosted by the Silicon Valley Cloud Computing Meetup.

Topic: Getting Friendly With Document Databases

Products covered: MongoDB ( and CouchDB (
Level: Introductory but not cursory. Full of examples.
Duration: 60 mins. (1 hour) — may have too much for an hour. Could do a bit more than an hour if need be.
Session Contents:
— Document databases
  • What are they?
  • Their essential structure (in the context of MongoDB and CouchDB)
  • Data types supported
  • Schemaless
— Creating, Reading, Updating and Deleting Documents
  • Using MongoDB
  • Using CouchDB
— Querying Documents
  • Filtering
  • Ordering
  • Limiting result set
  • Grouping
  • Joining (?)
(Includes MapReduce)
— Indexes
  • Types
  • How-to
— Very first steps in performance tuning
  • Understanding query plans
  • Faster query results
— A few peculiarities
— Questions
This should give you a head start but an hour isn’t enough to cover all the details so am planning on organizing a follow-up 2 day training in February. See you on Jan 24 at Fenwick & West.

Special Guest at the NIT PowerConnect

The NIT (National Institute of Technology) Almuni Network in the Silicon Valley organizes a monthly power connect networking event. I am honored to be invited as a special guest to their event this evening. I am not an NIT alumni (I attended St. Stephen’s College, XLRI and Courant, NYU) but do know that NIT (which was formerly known as REC) produces a number of very bright engineers every year. Although, IIT is the big global brand from India which has produced a number of very smart and well-recognized engineers, few know that NIT has a lot of great success stories as well.

I will be leading the conversation on NoSQL and Cloud Computing and would participate in a panel discussion on “Snakes and Ladders – How to climb the corporate Ladder” with a bunch of well-known and respected professionals including Bala Sahejpal (IT Director at Juniper Networks), Paul Chen (Director at PapayaMobile), Dilip Saraf (a career coach), Anand Kamannavar (Applied Ventures) and Biren Gandhi (Ex Studio CTO Zynga). Looking forward to the exciting event this evening. If you are coming to the event then look forward to seeing you there.

NoSQL Sessions at Silicon Valley Cloud Computing Meetup in January 2011

After Santa Claus has come and gone, NoSQL is coming to town! Come January 2011, I present the core NoSQL ideas, concepts, tools and technologies via a set of 4 day back-to-back sessions at the Silicon Valley Cloud Computing Meetup. The schedule is as follows:

Jan 24th (Monday): NoSQL Series – Part 1: Getting friendly with document databases
Jan 25th (Tuesday): NoSQL Series – Part 2: Nothing beats a distributed hash
Jan 26th (Wednesday): NoSQL Series – Part 3: HBase beyond the “Hello World!”
Jan 27th (Thursday): NoSQL Series – Part 4: Eventually it’s consistent
The venue is:
Fenwick & West
801 California St
Mountain View, CA 94041

Each day session starts at 7pm so you don’t have to miss work to join us for these sessions. Actually, wanted to make sure everyone was tired after a long day’s work so there were less questions :)

Each session is about an hour and a half long with a short break of 5 mins. or so in the middle. There is pizza, veggies and desserts to go along with the talk.

Thanks to Sebastian Stadil for organizing the Silicon Valley Cloud Computing Meetup and making these sessions possible. If you are into big data, cloud computing and web scale stuff and are in the bay area then this meetup is surely the one you should join.

All these talks will leverage my efforts towards writing Wiley’s Professional NoSQL (coming end of Q1/Q2 2011).

Come to Flash and the City 2010 in New York, NY

As I have said earlier, I am on a speaking engagement diet for a few months now and am sticking to a fewer number of conferences than I have in the last few years. The place I appear next is in a conference at my home base — New York City. Thanks to Elad Elrom, we have a great Flash conference, appropriately called — “Flash and the City“, right here in our city. Its a conference that has not only attracted a stunning lineup of speakers (including yours truly, just kidding :)) and includes some of the most relevant topics that would interest any and every Flash developer and architect but is also a conference that promises a lot of fun in the very amazing city. There are city tours, bar hopping events and dinner on the hudson. Doesn’t that sound attractive?

The event is scheduled to take place between May 13 and May 16 at the 3LD Technology Center in downtown Manhattan. You can look-up the schedule to get more details on who speaks on what and when.

My session titled  — “Flash amid the cool and the futuristic web” covers a number of different emerging aspects of the Flash platform that are shaping and will be relevant for the web of the future. So, if you care about the future of the web, do stop by.

See you at Flash and the City!

Speaking at 360 Flex San Jose — March 7-10, 2010

With the beginning of 2010 I have put breaks on my frequent speaking engagements. What I have decided, is to speak at select few events and venues, else for the most part keep to online communication. I will tell everyone about my plans for online sessions in a following post but for now let me talk more about the next event where I appear.

Going by the last few years, by the first week of March I would have spoken at atleast 4 to 5 events. However, this year is different and consciously so! My first event is 360 Flex at San Jose (March 7-10, 2010).  You may ask why 360 Flex and you may want to know what I am talking about at that conference. Let me first attempt to answer the “why” part of the question and then I will talk a bit about what I am presenting on at the conference.

360 Flex is one of the few conferences that is run by and for the developers. Its one big meetup for all those who are part of the Flex community. Its not a stage where companies pretend to demo the future. Its all about great content, friendly experience and immense value for your money. Its a place where even the expert takes a few tips back with him at the end of the conference. So it was not prudent for me to skip this event. Tom and John have been gracious to have me as a speaker for past many 360 events and I have enjoyed each one of them. So when I had to choose a select few, 360 Flex comfortably made the list for all the good reasons I enumerated earlier in this paragraph.

So now that I am going to 360 Flex, what am I going to talk about there. Answer: some regular Flex stuff hidden under a fancy topic! No, just kiding 😉 I speak about the new Multi-touch support in Flash Player 10.1 and AIR 2.0. I intend to talk about what this new support means and what it doesn’t and try and corroborate my statements with a few nice examples. It should be a good talk to attend even if you do not intend to build any touch screen applications in the near future. I will try and include a few essentials topics on touch screens and multi-touch as well to bring up to speed all those who haven’t had an exposure to this realm of application development.

If you are coming to 360 Flex, I would love to catch-up. If you haven’t registered yet, then without further adieu just follow this link to register for 360 Flex now.

Please lookup more about 360 Flex at — For schedule go to —

Before, I close, if at any time you were anxious why this speaking moritorium then here are my reasons —

  • Every time someone travels from coast to coast in the US, the person emits as much carbon dioxide as a full year of driving a car does. I would like to do my bit and contribute less to the global warming.
  • With the rise in the number of conferences and speakers, many events are becoming re-runs of the same old stuff, which you can easily Google for from the comfort of your home.
  • Its about time I started leveraging online media (especially video). I could reach a far greater number of people and help many more.
  • I am super busy with newer products in the making and customer work. Every time I am out I essentially slip down on my plans. I don’t want to continue to do that.
  • I am very keen on open spaces style events and advanced events as opposed to the 101 sessions. Honestly, 101 sessions are best delivered via online videos. So I think :)

Next Stop: NFJS Rocky Mountain Software Symposium

I speak on Flex and Java Integration, Flex and Hibernate and Collaborative real-time RIA this Sunday (November 22, 2009) at the No Fluff Just Stuff (NFJS) Rocky Mountain Software Symposium in Denver, CO. If you are in Denver and coming to the show, I hope to see you there.

Flex (Flash) Camp Wall Street Starts Tomorrow

Flex (Flash) Camp Wall Street starts tomorrow (November 16, 2009) in New York City. If you are coming to the event, do stop by to tell us about your exciting adventures in the world of RIA. Enjoy 14 exciting sessions over two days. Meet with the experts in the field. Mingle with the community and don’t forget to hang out in the after session sessions :) If you haven’t registered yet, don’t wait any longer as very few seats remain. Register online now at


(This cool speaker badge was created by Adam Flater as a draft initial artifact!)