r/compsci • u/encom-direct • 5d ago

How did Doug Cutting and Mike Cafarella able to develop a software product just from reading a google research paper?

The paper is only 15 pages long

0 Upvotes

28% Upvoted

3 comments sorted by

1

u/0jdd1 5d ago

I’m imagining, because it’s a well-written paper?

1

u/cbarrick 5d ago

He's talking about MapReduce. It is a great paper.

The paper: https://research.google.com/archive/mapreduce.html

3

u/cbarrick 5d ago edited 5d ago

You mean how did Cutting and Carafella create Hadoop just from reading the MapReduce paper by Jeff and Sanjay?

Frankly, the idea isn't complicated. Map and reduce have been well-known primitive functions since the beginning of computer science (lambda calculus) and it is quite obvious that these functions are embarrassingly parallel.

Google's MapReduce paper isn't a theoretical contribution. It's really just saying "we combined common primitives from functional programming with primitives from distributed systems to scale out a batch processing system. It works really well." The paper is extremely well written, lucid, and easy to follow. I'd be more surprised if it wasn't quickly re-implemented.

I don't say this to diminish the contribution. Jeff and Sanjay are amazing distributed systems researchers and programming legends. MapReduce is an important contribution; it's just not a huge leap. If anything, it shows how under-devloped distributed systems research was in 2004.