CS 442 Projects

Projects are to be performed individually, or in groups of 2 or 3. If the project is to be performed in groups, then the scope should be larger. Also, group projects should clearly specify who is going to do what part of the project in the project definition.

The goal of the project is to get some implementation experience with respect to distributed systems. There are way too many technologies out there that are related to distributed systems. Your project may take various different forms, such as:

• Using a distributed systems technology to implement a non-trivial application
• Implementing a simple middleware that provides a distributed systems abstraction

Some example technologies of interest:

• RPC/RMI
  • Application: Implement a non-trivial application that uses these technologies
  • Middleware: Implement an RPC system on top of sockets (including stub and skeleton generators)
• Distributed shared memory (DSM)
  • Middleware: Implement DSM (contact me if you want to do this, but don't know where to start)
• MPI
  • Application: Implement a parallel algorithm using MPI (must show good scalability)
  • Middleware: Implement a simple MPI middleware (only a small subset of the MPI abstractions)
• Key-value stores (memchached (in memory), HBASE (disk-backed), etc.)
  • Middleware: Implement a basic distributed key-value store (in-memory).
• Distributed hash-tables (Chord-like)
  • Application: Build a decentralized application that relies on DHTs
  • Middleware: Implement a basic distributed hash-table
• Map/Reduce
  • Application: Implement a distributed/parallel application using M/R (should scale well)
  • Middleware: Implement your own M/R runtime
• Stream processing (Twitter Storm, Yahoo! S4, IBM Streams (contact me to get a student copy))
  • Application: Implement a distributed stream processing application (should work with a real-time source)
• BSP-style graph processing (GoldenORB)
  • Application: Implement graph algorithms on BSP middleware, using vertex-based programming
  • Middleware: Implement a BSP middleware
• Grid computing (SETI@Home like)
  • Middleware: Implement a middleware for distributing independent tasks
• Lowwer-level abstractions
  • Implement a distributed lock server
  • Implement a time synchronization service
  • Implement a library for the Paxos protocol

You can pick something else as well. Note that doing a middleware project is often more difficult but you will learn more. It is important to scope such projects properly.

We have two deliverables for the project:

• A report describing the project: requirements, the internal design, and the results. For projects that require showing performance and/or fault-tolerance results, a detailed evaluation is needed as part of the report.
• The code and the working demo.

Important Dates:

• A single page project description: Due Week 4, Wednesday
• Report and demos: Due last week of the course