COMP90015 Distributed Systems Project 2 - Decentralized Chat Aaron Harwood School of Computing and Information Systems © The University of Melbourne 2021 Semester II Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 1 / 33 Synopsis • In this assignment you will transform your existing chat program into a decentralized chat application. • You may reuse as much of your project 1 code as you wish. You may rewrite everything if you wish as well. Feedback from project 1 assessment can be used to help improve your project 2 code and report. • The basic operation and protocol from project 1 will be the same in project 2, however instead of a client and server there will be only one process which we will call a peer. The peer will act as both a client and a server in the decentralized system. • The intended operation of the chat peer is described in this slide set. • You are required to also implement one extended feature as described next. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 2 / 33 Extended Feature I As well as the base protocol that everyone must implement, each group must choose one extended feature from the list of three choices below, design the protocol as an extension to the base protocol and implement in their peer: • Room Migration – Rooms can be moved from one peer to another. E.g. before a peer quits it can send its rooms to other peers. All peers that are currently in those rooms should automatically disconnect and reconnect to the room when it has been relocated. The peer that receives a room becomes the owner of that room. Selection of which peers to send rooms to should be based on ensuring that all existing peers in those rooms can connect to the new peer where their room is migrated to. Rooms should not become “lost”, e.g. if the peer that is receiving a room also quits in the middle of receiving it and therefore does not migrate it further. Of course, if there are no neighboring peers to migrate a room to then the room is lost if the peer quits. The user of a peer should also be able to issue a command to migrate a room on a room-by-room basis. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 3 / 33 Extended Feature II • Shout – A Shout command can be used by any user currently joined in a room. The command causes a provided message to be delivered to all rooms on all peers of the network. All peers in the network includes all peers for which there is a path of connections from the shouting peer, not just those peers that are directly reachable by the shouting peer. The order of delivery at all rooms on all peers of the network must be FIFO, i.e. that the order of the messages shouted by a given user matches the order of those messages delivered at each room at each peer, regardeless of whether some peers have connected and/or disconnected in between. When the shouted message is delivered to a room, the identity of the shouter should be listed as “IP:port shouted”, where IP and port are the values as seen by the peer hosting the room that the shouting peer is currently in. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 4 / 33 Extended Feature III • File Attachment – Attach a file or files to the message. Users should have the option to download specific attachments or download all attachments for a given received message. The download should happen by one of two mechanisms: (i) directly connecting to the peer that sent the message, if the peer indicates (as metadata in the message) this is allowed (e.g. the peer would typically have a public IP address for this to work), or else the peer that sent the message uploads the attachments to the peer maintaining the room and peers receiving the message download it from that peer. When a peer is uploading or downloading files it should not block the user from issuing further commands. Note: be carefull when implementing functionality that access the local file system - this creates a strong potential vulnerability. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 5 / 33 Chat Peer The base functionality that all groups must implement is briefly: • Each peer is started by a user who interacts with the peer on the command line in the same way as interacting with the client in project 1. • Each peer will maintain the chat rooms that are created by its user. Users can only create rooms on their own peer. As explained shortly, the create and delete room commands don’t require message passing because they are only executed locally. • There is no “main hall” room in project 2. A peer that connects to another peer must first obtain a list of rooms and then join one. The exact session protocol is discussed further in. • Peers will provide a list of neihboring peers with IP address and port number, that allows other peers to search the network of peers for chat rooms, using an iterative search mechanism. • Peers will allow other peers to connect to them, to search for chat rooms and to join chat rooms and chat in them. If a peer does not have a public IP address then only peers on the same private network as the peer can connect to it. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 6 / 33 Overview of peer protocol I • Protocol aspects taken from Project 1: • List – (same as proj1) respond with a list of rooms that the peer is maintaining • Join – (same as proj1) allow a peer to join a room • Who – (same as proj1) provide a list of who is in the room • Kick – (similar to proj1) the kicked user is forcibly disconnected from the peer, and blocked from reconnecting. This is effectively a local command that does not require message passing, as explained below, since only the owner of a room can kick others and rooms created on a peer can only be created by the user of that peer. • Quit – (same as proj1) disconnect from the peer • Message – (same as proj1) the message is sent to all peers in the room • Protocol aspects that are very different to Project 1: • The identity protocol and associated messages from project 1 will not be used. Rather a peer’s IP and port number combination, e.g. 192.168.1.10:3000, as seen by the peer hosting the room, will be used and shown as the peer’s identity. The peer cannot change its identity unless it disconnects and reconnects from a different IP address and/or a different port number. • The CreateRoom functionality will exist but will only be accepted if the command was given by the user of the peer, which means that effectively no actual messages are required since this command will never be accepted by any other peers. It can therefore be directly implemented in the peer without message passing. • The Delete functionality will exist but similar to CreateRoom, the delete command is a local command only that does not require message passing. • Additional protocol aspects: Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 7 / 33 Overview of peer protocol II • ListNeighbors – a peer can ask a peer for a list of peers’ network addresss that are connected to it (of course, connected peers that are joined in a room also have their network address exposed however not all connected peers must be joined in a room) • SearchNetwork – a local command that causes the peer to do a breadth first search of all peers available to it (using a combination of ListNeighbors and RoomList messages, to gather a list of chat rooms over all accessible peers and show that to the user • Connect – a local command that causes the peer to make a TCP connection to the peer as given by the user, i.e. the user gives the IP and port number • Help – a local command that lists all of the commands available to the user with their syntax that shows how to use them Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 8 / 33 Initial connection to a peer From this point on, for the purposes of discussion a peer that is connecting to another peer will be called the client and the other peer will be called the server. • When a peer process is started by the user, it does not connect to any other peer initially. • The user must use the Connect command (local command) to cause the peer to make a TCP connection to another peer. The user must know the IP and port number of the peer to be connected to. • When the connection is established from client to server, the server simply waits for the client to send a command. • While connected: • The user can use the List and other commands to find rooms, join a room and start chatting, very much the same way as project 1. The biggest differences are that the user’s identity is its IP and port number (as seen by the server) and that there is no “main hall”. • The user can use the ListNeighbors and SearchNetwork command (the operation of these commands is explained further in) to search the broader network of peers. • The user must use the Quit command to disconnect from the peer. The user can then connect to another peer. • When not connected to another peer, all of the commands issued are with respect to the local peer’s room list, including the local commands CreateRoom, Delete and Kick. • The peer process quits (disconnecting any connections first) when the standard input is closed, using Ctrl-D on the terminal. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 9 / 33 Client Command Line Interface • The client must accept input from standard input. Each line of input (terminated by a newline) is interpreted by the client as either a command or a message. If the line of input starts with a hash character “#” then it is interpreted as a command, otherwise it is interpreted as a message. • The client must write all output to standard output. Since chat is asynchronous, messages may arrive at any time and will be written to standard output when they arrive. For an extended feature the command line interface should provide ways to interact with the extended feature. • If some data such as streaming video needs to be displayed, a GUI window can be opened to display that. It should be controllable from the command line and may also be controllable from the GUI window itself. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 10 / 33 JSON Format • All protocol messages, i.e. sent between client and server, will be in the form of newline terminated JSON objects. • A JSON library must be used to marshal JSON output and to unmarshal JSON objects. Do not implement JSON (un)marshalling yourself. • All data written to the TCP connection must be UTF8 encoded. • When implementing an extended feature you may dynamically change to another data format other than JSON for that feature if you wish, but the base protocol must be implemented in JSON and the base protocol must continue to work regardless of whether the feature is used or not. E.g. you may even use a separate TCP connection to transmit binary data when required and have additional default ports for those aspects. You may add some fields to the base protocol messages if you need to support your extended feature, but you cannot change any of the existing fields. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 11 / 33 Connect command The peer to connect to is not given on the command line. Rather to make a connection to another peer: >#connect 142.250.70.238:4444 [] 192.168.1.10:38283> • The room is empty [] (no room) and the identity is the IP and port number as seen by the remote peer. • Note this example is a little contrived: the peer with the public IP address 142.250.70.238 would not see the private IP address 192.168.1.10, but rather would see the public IP address of the router that the privately connected peer is getting access through to the public network. Your actual output will differ according to the specific networks that you test on. The examples in this project specification continue with these contrived IP addresses. • If the port number is not provided then the default port number is used. As well, the connect command should allow the user to optionally specify which port they would like the TCP connection to use locally when making the connection: >#connect 142.250.70.238:4444 5000 [] 192.168.1.10:5000> Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 12 / 33 Help command The help command should just list the available commands and their syntax, e.g. like: >#help #help - list this information #connect IP[:port] [local port] - connect to another peer #quit - disconnect from a peer ... You can simply provide a short description of all the commands like above. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 13 / 33 Join Room Protocol The client may at any time send a Join message to the server to indicate that the client wishes to change their current room to the indicated room. { "type":"join", "roomid":"comp90015" } E.g.: [] 192.168.1.10:38283>#join comp90015 Note the format above shows [] to indicate that the client having identity 192.168.1.10:38283 as seen by the server is not in a room. We can use the empty string "" to represent not in a room. Text messages sent when not in a room are ignored. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 14 / 33 Join Room Protocol When receiving a Join message the server must follow exactly this protocol: • If roomid is invalid or non existent then client’s current room will not change. The special room given by "" indicates that the client wants to leave the room but not join another room. I.e. stay connected but not be in any room. • Otherwise the client’s current room will change to the requested room. • If the room did not change then the server will send a RoomChange message only to the client that requested the room change. • If the room did change, then the server will send a RoomChange message to all clients currently in the requesting client’s current room and the requesting client’s requested room. E.g.: { "type":"roomchange", "identity":"192.168.1.10:38283", "former":"", "roomid":"comp90015" } Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 15 / 33 Join Room Protocol The client will determine from the message whether the request was successful or not and will output relevant information: • If the request was not successful: “The requested room is invalid or non existent.” • If the request was successful then either “192.168.1.10:4444 moved to comp90015” if the client was not already in a room or “192.168.1.10:4444 moved from roomid to comp90015” if the client was already in a room called roomid in this example. And on the command line: [] 192.168.1.10:38283>#join comp90015 [] 192.168.1.10:38283> 192.168.1.10:38283 moved to comp90015 [comp90015] 192.168.1.10:38283> Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 16 / 33 Room Contents Message I The RoomContents message lists all client identities currently in the room: { "type":"roomcontents", "roomid":"comp90015", "identities":["192.168.1.10:38283","192.168.1.10:5000","142.250.70.238:4444"], } There is no need to indicate an owner since the owner of the room is always the peer that is sending the message. The client may at any time send a Who message to request a RoomContents message from the server for a given room: { "type":"who", "roomid":"comp90015" } Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 17 / 33 Room Contents Message II e.g. [comp90025] 192.168.1.10:38283>#who comp90015 Note that obviously the server knows the identity of every client connection, so the identity of the sender is not required. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 18 / 33 Room List Message The RoomList message lists all room ids and the count of identities in each room: { "type":"roomlist", "rooms":[{"roomid":"comp90025","count":5}, {"roomid":"comp90015","count":7}, {"roomid":"CovidWillComeToAnEnd","count":4}] } The client may at any time send a List message to request a RoomList message from the server: { "type":"list" } Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 19 / 33 Room Information at the Client The client displays information regarding rooms and room lists whenever they arrive. Examples have been given on previous slides. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 20 / 33 Create Room Only the user of the peer can create a room on that peer, using the CreateRoom command. There is no message associated with this command. Rather the logic is implemented locally and is the same as in project 1: • If the requested name of the room is valid and does not already exist then it is created – the room name must contain alphanumeric characters only, start with an upper or lower case letter, have at least 3 characters and at most 32 characters. • The peer outputs either e.g. “Room jokes created.” or “Room jokes is invalid or already in use.” depending on whether the local command worked or not, for the example “jokes” room. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 21 / 33 Room Ownership The owner of a room is always the peer that created it. It can only be created by the user of that peer. Therefore there is no metadata required to maintain the owner of each room. For the purposes of discussion, the “owner” of a room always refers to the peer that created it. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 22 / 33 Delete Room The owner of a room can at any time issue a Delete command. Similarly to the create room and kick user, this command does not generate messages and the logic is simply executed locally. The peer will first treat this as if all users of the room had sent a RoomChange message to the empty room "". Then the peer will delete the room. The peer can print whether the room was successfully deleted or not. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 23 / 33 Messages The client can at any time send a Message: { "type":"message", "content":"Hi there!" } A message received by the server is relayed to all clients that are within the same room as the client who sent the message. The id of the sender is appended to the message: { "type":"message", "identity":"192.168.1.10:38283", "content":"Hi there!" } Clients will never receive messages for rooms that they are not in and messages sent by a client that is not in a room are ignored (silently dropped). Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 24 / 33 Quit The client can at any time send a Quit message: { "type":"quit" } The server will remove the user from their current room, sending an appropriate RoomChange message to all clients in that room. The roomid of the RoomChange message will be an empty string. When the server sends the RoomChange event to the disconnecting client, then it can close the connection. When the client that is disconnecting receives the RoomChange message, then it can close the connection. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 25 / 33 Abrupt Disconnections If a client gets disconnected (e.g. if the TCP connection is broken), then the server treats this as if the client had sent a Quit message. The server should send appropriate messages to other clients in the system and set room ownership appropriately. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 26 / 33 List Neighbors Protocol A client may request the server to list its neighbors by sending a list neighbors message: { "type":"listneighbors" } The server responds with a list of peers that are currently connected (not including its own network address, or the address of the client that issued the request): { "neighbors":["192.168.1.10:5000","142.250.70.238:4444"] } The list may be empty. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 27 / 33 Search Network • The SearchNetwork command is a local command. When the user issues this command, the peer is required to automatically crawl over all of the accesible peers in the network (using a breadth-first recursive strategy), connecting to each peer (using a connection that is separate to the peer’s existing connection if there is one) and find the rooms available in each peer using a List command and other peers to search using the ListNeighbors request. The output should be a room list for each peer that is found, e.g.: [] 192.168.1.10:38283>#searchnetwork [] 192.168.1.10:38283> 192.168.1.10:5000 gameworld 10 users 142.250.70.238:4444 googlehut 52 users alumni 20 users 104.215.148.63:4444 microsoftroom 28 users • In this example, the peer 104.215.148.63:4444 was discovered after listing the neighbors of one of the peers that was already known. The user can then decide whether to manually connect to one of these other peers or not, using the Connect command. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 28 / 33 Technical aspects • Use Java 1.11. • All message formats should be JSON encoded. Use a JSON library for this. The format can be different for the extended feature. • Your program should be cleanly finished by terminating all running threads. • Package everything into a single runnable jar file, for the peer. • Your peer should be executable exactly as follows where the port option -p gives the port that the peer listens on for incomming connections and the port option -i gives the port that the peer uses to make connections to other peers when the user issues the Connect command: $> java -jar chatpeer.jar [-p port] [-i port] • Use command line option parsing (e.g. using the args4j library or your choice). • The default server port listened on should be 4444. A command line option can override this. The default port used to make connections to other peers when the user uses the Connect command can be whatever port is used by the OS (i.e. an emphemeral port), and therefore may change each time the user disconnects and reconnects. • Terminating the standard input (Ctrl-D) should terminate the peer. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 29 / 33 Your Report I Only 1 report is required per team. • Use 10pt font, double column, 1 inch margin all around. Put both team members’ login names and student numbers at the top of the report. • In a few sentence, include a statement of the contribution from each team member. • In about 1000 words: Discuss your design and implementation of your chosen extended feature. Use formal diagrams to help with your explanation. Provide an analysis/critical discussion of your extended feature implementation with respect to relevant selected challenges of distributed systems. • In about 500 words: Develop/discuss a model that describes the decentralized chat application, which includes your extended feature, in terms of the communication patterns and network topology that arises over time as new peers connect to exsisting peers and old peers disconnect, and with respect to different commands that users can issue. E.g. a peer can only have one existing connection to another peer at any point in time, which limits the kind of patterns that we can see, other than when e.g. a breadth first search is being conducted, which is limited by reachability between machines. Discuss all of these aspects with respect to your model. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 30 / 33 Your Report II • In about 500 words: Develop a failure model for the decentralized chat application which includes your extended feature, using basic assumptions as discussed in the lecture, including e.g. assumptions of failure rate of a peer process. Use the model to make some basic statements about how fault tolerant the decentralized chat application is (or is not as the case may be) to process failure and network failure. You may consider what happens when there are N peers connected into one large connected component and N becomes very large. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 31 / 33 Assessment • Assessment for this project is worth 20% of the total subject assessment. • The project assessment is split between the software implementation (50%) and the report (50%): • Software assessment is comprised of two parts: • Up to 65% of software marks for an implementation that compiles to provide a chat peer jar file, as also submitted, that provides the minimal amount of functionality: client can connect and join a room, can send and receive chat messages and can quit. If this minimal functionality does not work then at most 65% will be awarded for the software, based on what aspects do appear to work and/or code inspection. • Additional 15% of software marks are awarded based on correct implementation of remaining base protocol, including ListNeighbors and SearchNetwork. These marks are only awarded if the functionality above all works as expected. • Final 20% of the software marks are awarded based on a correct implementation of the extended feature. • When some part of an implementation does not work, up to 25% of the marks for that part of the implementation may be awarded based on code inspection. In this case, well presented code has a better chance of attracting more marks. • Report: • Assessment is based on presentation, demonstration of critical thinking, correctness, being concise, consistent in terminology, and complete in addressing the required criteria on the previous slide. • Assessment is weighted based on the word limitations for each criteria. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 32 / 33 Submission You need to submit the following (only one person needs to submit per team) via LMS: • Your report in PDF format only. • Your chatpeer.jar file. • Your source files in a .ZIP or .TAR archive only. Submissions will be due on Friday, Week 12, midnight. Submissions will be via LMS and more details will be given closer to the due date. Aaron Harwood (School of Computing and Information Systems © The University of Melbourne)COMP90015 Distributed Systems 2021 Semester II 33 / 33
欢迎咨询51作业君