Monthly Archives: January 2011

Research Papers and Videos on Google Bigtable, GFS, Chubby and MapReduce

Thanks all for coming to my talk today on sorted ordered column-family stores at the Silicon Valley Cloud Computing Meetup. Here are the links to the Google’s research papers and a couple of videos that relate to Bigtable (and Google App Engine internals):

Bigtable: A Distributed Storage System for Structured Datahttp://labs.google.com/papers/bigtable.html

The Chubby Lock Service for Loosely-Coupled Distributed Systems

http://labs.google.com/papers/chubby.html

The Google File System

http://labs.google.com/papers/gfs.html

MapReduce: Simplified Data Processing on Large Clusters

http://labs.google.com/papers/mapreduce.html

BigTable: A Distributed Structured Storage System –

http://video.google.com/videoplay?docid=7278544055668715642&hl=en#

Google I/O 2008 – App Engine Datastore Under the Covers –

http://www.youtube.com/watch?v=tx5gdoNpcZM

Enjoy reading the research papers and watching the videos!


Getting Friendly With Document Databases

Here is a draft of what I plan to present in Part 1 of the NoSQL Series on Jan 24th at Fenwick & West in Mountain View, CA (The series has 4 parts in all. It runs between 1/24 and 1/27, everyday at 7pm). The event is  hosted by the Silicon Valley Cloud Computing Meetup.

Topic: Getting Friendly With Document Databases

Scope:
Products covered: MongoDB (mongodb.org) and CouchDB (couchdb.apache.org)
Level: Introductory but not cursory. Full of examples.
Duration: 60 mins. (1 hour) — may have too much for an hour. Could do a bit more than an hour if need be.
Session Contents:
– Document databases
  • What are they?
  • Their essential structure (in the context of MongoDB and CouchDB)
  • Data types supported
  • Schemaless
– Creating, Reading, Updating and Deleting Documents
  • Using MongoDB
  • Using CouchDB
– Querying Documents
  • Filtering
  • Ordering
  • Limiting result set
  • Grouping
  • Joining (?)
(Includes MapReduce)
– Indexes
  • Types
  • How-to
– Very first steps in performance tuning
  • Understanding query plans
  • Faster query results
– A few peculiarities
– Questions
This should give you a head start but an hour isn’t enough to cover all the details so am planning on organizing a follow-up 2 day training in February. See you on Jan 24 at Fenwick & West.

Special Guest at the NIT PowerConnect

The NIT (National Institute of Technology) Almuni Network in the Silicon Valley organizes a monthly power connect networking event. I am honored to be invited as a special guest to their event this evening. I am not an NIT alumni (I attended St. Stephen’s College, XLRI and Courant, NYU) but do know that NIT (which was formerly known as REC) produces a number of very bright engineers every year. Although, IIT is the big global brand from India which has produced a number of very smart and well-recognized engineers, few know that NIT has a lot of great success stories as well.

I will be leading the conversation on NoSQL and Cloud Computing and would participate in a panel discussion on “Snakes and Ladders – How to climb the corporate Ladder” with a bunch of well-known and respected professionals including Bala Sahejpal (IT Director at Juniper Networks), Paul Chen (Director at PapayaMobile), Dilip Saraf (a career coach), Anand Kamannavar (Applied Ventures) and Biren Gandhi (Ex Studio CTO Zynga). Looking forward to the exciting event this evening. If you are coming to the event then look forward to seeing you there.

5 Technology Application Trends in 2011!

5 technology application trends that I think will be most popular in 2011 are:

  1. Tablets, tablets and tablets: iPad started the fever and its not stopping anytime soon. Newer and smarter tablets of all sizes, form factors and capabilities will emerge. Newer and newer applications for these devices will be available.
  2. Big data will get bigger: More and more big data will become available in the public domain and we will see the emergence of newer and smarter storage and analytics solutions in the space. In other words NoSQL and all tools that help manage big data will boom. Cloud will continue its expansion.
  3. Local will be king: Groupon has show the way but there is a lot more to win! Hyperlocal communities will be the way forward. You will see a lot more startups in the space. If you are an investor, don’t forget to put some money there :)
  4. Social networking shakeout & correction: Every boom meets a correction, Facebook and friends will see some correction as well.
  5. Growth of collaboration: the audio-video segment has been largely a consume only space for a while. Collaborative rich communication and interaction will see some innovative new applications.

Happy New Year!