Lab 1: Peer-to-Peer File System CSE 513 - Distributed Systems The Pennsylvania State University Fall 2019 Yeming Tang
[email protected] Due Date: Oct 10, 2019 1 Introduction In this lab, you will design and implement a simple peer-to-peer (P2P) file sharing system. The goal of the assignment is to provide experience on implementing a P2P network. 2 System Description The system will include multiple clients (also named peers) and one central server. A peer can join the P2P file sharing system by connecting to the server and providing a list of files it wants to share. The server shall keep a list of all the files shared on the network. The file being distributed is divided into chunks (e.g. a 10 MB file may be transmitted as ten 1 MB chunks). For each file, the server should keep track of the list of chunks each peer has. As a peer receives a new chunk of the file it becomes a source (of that chunk) for other peers. When a peer intends to download a file, it will initiate a direct connection to the relevant peers to download the file. A peer should be able to download different chunks of the file simultaneously from many peers. 3 Protocol Requirements There are many ways to approach this project. At its core is a messaging system amongst peers and between peers and the server. You may design the protocol in any way you wish as long as it fulfills the system description and requirements. Below is a minimal list of messages you can use. Feel free to add more messages if you like; however if you design your project carefully, these are the only messages you will need for a minimal yet successful design. 1 Peer Request Server/Peer Reply Register Request: Tells the server what files the peer wants to share with the network. Takes in the IP address (uint32) and port (uint16) for the end- point to accept peer connections for download; the number of files to register (uint16); and for every file, a file name (string) and its length (uint32). Register Reply: For each file, it advises if the file registration was a success (Boolean). File List Request: Asks the server for the list of files. File List Reply: Includes the number of files in the list (uint16); and for each file, a file name (string) and a file length (uint32). File Locations Request: Asks the server for the IP endpoints of the peers containing the requested file (string). File Locations Reply: Includes number of endpoints (uint16); then for each endpoint, chunks of the file it has, an IP address (uint32) and port (uint16). Chunk Register Request: Tells the server when a peer receives a new chunk of the file and becomes a source (of that chunk) for other peers. ChunkRegister Reply: Advises if the chunk registration was a success (Boolean). File Chunk Request: Asks the peer to return the file chunk. Reads in a file name (string), chunk in- dicator (uint32). File Chunk Reply: A stream of bytes repre- senting the requested chunk (array of bytes). 4 Basic Requirements All projects must fulfill the following requirements: • Multiple Connections: Your peers and servers must be able to support multiple connections simultaneously. You cannot delay any message arbitrarily because of lack of parallelism. (Hint: use multithreading or select() or epoll().) • Parallel Downloading: Your system should maximize its download speed by receiving a file from multiple peers and assemble the file. For example, before downloading, the client can get the file size from a server and be able to divide the file into several chunks to be downloaded from multiple peers simultaneously. • Chunk Selection: When a peer downloads chunks from another peer, it should have an option of using ”rarest first” to determine which chunk to download. In the demonstration, your program should be able to output which chunks of the file its peers have, and from which peer it downloads the chunks of the file. • Chunk Download Completion: Upon finishing the download of a chunk, a peer must reg- ister that chunk with the server to become a source (of that chunk) for other peers. 2 • Failure Tolerance: Your programs must not crash if a peer or the server unexpectedly fails or leaves the network. When the peer recovers, it should be able to join the network and resume uploading and downloading. 5 Group Requirements If you wish, you may form a group with at most one other person. The following requirements are optional for individual work but required for group work: • Incentive: A peer sends chunks to four peers currently sending it at the highest data rate, and choke others. Every 30 seconds, randomly select another peer to optimistically unchoke it. For demonstration purpose, you may choose different parameters. 6 Interface Requirements The user should be able to specify which files t o s hare a s c ommand l ine a rguments. You may specify a directory or a list of files. You must be able to ask for the file list and choose a file to download. You must be able to view progress on active downloads. 7 Programming Language Requirements You may implement this project in the language and operating system of your choice (but no script- ing languages) as long as you can provide a live demo. Most people will benefit from developing this project in C under a UNIX environment. Other choices include C++, C#, or Java. Some platforms are easier than others so choose wisely. 8 What and How to submit Your submission should include: 1. A detailed description of the system you built, along with protocol specifications. 2. A description of the structure of your program. 3. A description of which part works and which part doesn’t. 4. A sample output of your program. Please notate the output so that it can be understood easily. 5. Well-commented source code. 6. A system description with makefile or compile command. Note, we only accept documents in pdf or txt format. Then, 3 1. Compress all above materials in a tar file or zip file named lastname firstname 1.zip (or .tar). 2. Submit it in attachment to Yeming Tang (
[email protected]) and the subject should be “CSE 513 - Lab 1 Submission”. 3. Please prepare a demo for your project. 9 Resources • You can use machines in your own lab or machines in the second floor of Westgate building such as W204 or W205. You can also “ssh” to the machines at other locations. • You need to use socket programming to finish this lab. You can search for ”socket program- ming” to be familiar with it. • The following books might be helpful for this assignment. – “Unix network programming” by W. Richard Stevens – “Internetworking with TCP/IP” series by Comer. • Compiling C programs under Solaris gcc -lsocket -lnsl -o
