Cardiff School of Computer Science and Informatics
Coursework Assessment Pro-forma

Module Code: CMT206
Module Title: Human Centric Computing
Lecturer: Dr Alia Abdelmoty, Dr Daniel J. Finnegan, Dr Hantao Liu
Assessment Title: Evaluation And Analysis Of Interface Design
Assessment Number: 1
Date Set: Monday 22 February 2021 (Week 1)
Submission Date and Time:
Assignment 1 (Q1): 26 March 2021 at 9:30am (Week 5)
Assignment 2 (Q2): 7 May 2021 at 9:30am (Week 8)
Assignment 3 (Q3): 28 May 2021 at 9:30am (Week 11)
Return Date:
Assignment 1 (Q1): 14 May 2021
Assignment 2 (Q2): 4 June 2021
Assignment 3 (Q3): 25 June 2021

This assignment is worth 100% (Q1: 33%, Q2: 33%, Q3:34%) of the total marks available for
this module. If coursework is submitted late (and where there are no extenuating
circumstances):

1 If the assessment is submitted no later than 24 hours after the deadline,
the mark for the assessment will be capped at the minimum pass mark;
2 If the assessment is submitted more than 24 hours after the deadline, a
mark of 0 will be given for the assessment.

Your submission must include the official Coursework Submission Cover sheet, which can be
found here:

https://docs.cs.cf.ac.uk/downloads/coursework/Coversheet.pdf

Submission Instructions

Description Type Name
Cover sheet Compulsory One PDF (.pdf) file per assignment Q1_[student number].pdf
Q2_[student number].pdf
Q3_[student number].pdf
Q1 Compulsory One Word (.docx) file
One spreadsheet (.xslx) file
Q1_[student number].docx
Q1_[student number].xslx
Q2 Compulsory One Word (.docx) file Q2_[student number].docx
Q3 Compulsory One Word (.docx) file Q3_[student number].docx

Any deviation from the submission instructions above (including the number and types of
files submitted) will result in a mark of zero for the assessment or question part.
Staff reserve the right to invite students to a meeting to discuss coursework submissions

Q1

Virtual meeting applications (such as Zoom, Teams, Webex, Blackboard collaborate, etc.)
have become essential working and social communication tools. There are many
applications on the market, and all compete for users.
In this assignment you will consider the design of one of these tools and write a heuristic
evaluation report on the usability of its interface.
Q1. Part 1: User and Task Analysis (10 marks)
a. Identify and describe One primary Persona for your application. Include all
components of the persona description, as discussed in the lectures.

Note: Applications can serve a wide variety of users and as such, may have multiple
primary personas. The persona you choose will guide your consideration for the
subsequent steps in this exercise. ((3 out of 10 marks))
b. Identify and describe Two primary tasks/use cases in the system (important and
frequently used functions by your primary persona). Provide a brief (text)
description of the task/use case and the outline of the steps in the basic flow
scenario and the alternative flow scenarios of the use case. ( (3 out of 10 marks))

Note: Tasks such as Register/Login are not primary use cases. These are
supplementary functions to allow a user to access the primary function of the
system.
c. Produce Two State Transition Networks (STNs) to describe the interaction flow in
your use case scenarios (for the basic and alternative flows in each use case).
Use an abstract depiction of the screens with no detail on design elements of the
interface. Your STNs need only depict states and user actions between states in the
use case scenarios. ((4 out of 10 marks))

Note: Generally speaking, an interface state changes when an event occurs. For
example, an entry of text in a text field, a click of a button, etc. Your STNs need to
depict all possible states of interaction in the use case scenarios.
Q1. Part 2: Heuristic Evaluation (10 marks)
For every use case, do a heuristic evaluation of the individual states of the interface,
identifying all usability issues.
Report on the usability issues in an objective manner and present the issues in a
systematic and standard format using the templates (Excel spreadsheet) provided and
discussed in the lectures. Your report should include,
a. the usability problems you identified with some justification of the severity score
assigned to the problems and your suggestions for addressing the issues you
identified in subsequent revisions of the interface, (5 out of 10 marks)
b. the good design features that should be maintained in any subsequent revision of
the application. (5 out of 10 marks)
Learning Outcomes Assessed
1. Understand the complex nature of users and apply heuristics to create and evaluate
inclusive and multimodal user experiences.
2. Evaluate competing proposals for interface design and implementation.
3. Apply human centric design methodologies in the context of current and emerging
interaction technologies such as virtual and augmented reality.
Criteria for assessment

Credit will be awarded against the following criteria.
Component &
Contribution
Fail < 50% Pass (50-59%) Merit(60-69%) Distinction >= 70%
Q1 Part 1(a) (3
marks)
Personas are missing
or not clearly justified
for the application.
Suitable choice of
Personas to represent
user segments
Sketchy description of
Personas that will not
support its utility for
design.
Persona described in
general terms, but is
not specific enough to
clarify its specific
needs from the
application.


Persona explained
clearly: Which
functions of the
application are
important to this
person?
How (context,
platform) and
When (how often) will
this person use the
application?

Q1 Part 1(b) (3
marks)
Use cases are not
described.


Good choice of
primary use cases,
but may not be
optimal primary
functions for the
chosen persona.

Brief descriptions
provided of one or
both use cases, but
basic and alternative
scenarios are not
clear or complete .

Good choice of
primary use cases
that are relevant to
the chosen primary
persona

Good description of
both use cases with
clear and complete
outline of basic
scenarios.
Use cases are
relevant to the
persona and their
primary status justified
to the needs to the
persona.

Excellent account of
both use cases and
complete account of
basic and alternative
scenarios.
Q1 Part 1( c) (4
marks)
STNs are missing or
do not relate to
identified use case
scenarios.
STN includes most of
possible states of the
interface, but some
states may be
missing.
Some user actions
are annotated.
Some steps of the
use case scenario are
not included in the
STN.
STN includes all
possible states of the
interface.
User actions are
mostly included, but
some may be missing
STN accounts for all
steps of the use case
scenario from start to
end.
STN includes all
possible states of the
interface.
User actions are clear
and completely
describe the transition
needed between
states.
STN accounts for all
steps of the use case
scenario from start to
end.
Q1 Part 2 (10
marks)
HE misses most of
the usability issues, is
not consistently or
exhaustively applied.

Mention of good
usability features is
patchy across the
interface.
.
HE considers the
usability of some
aspects of the
considered states, but
there are obvious
omissions of major
usability issues.

Usability problems are
recorded in an
adequate but
sometimes
inconsistent format.

Mention of good
usability features is
patchy across the
interface.

HE considers the
usability of most
aspects of the
considered states.

Usability problems are
recorded in a
consistent format.

Good usability
features are mostly
explored, but not
exhaustively reported.

HE considers the
usability of all aspects
(selection of interface
components, layout,
error handling,
feedback, etc.) of the
considered states.

Usability problems are
recorded in an
consistent format and
is complete, including,
severity rating and
possible solutions.

Good usability
features are explored
fully and documented
appropriately.


Feedback and suggestion for future learning

Feedback on your coursework will address the above criteria. Individual feedback and marks
will be returned on 14 May 2021 via Learning Central.

Feedback from this assignment will be useful for your dissertation.

Q2

This assignment is divided into 2 parts. Parts 1 and 2 are completed individually. Part 1
requires you to plan and conduct an experiment choosing from one of 3 problems. Part 2
involves reporting and reflecting on your experiment. For this assignment, you must submit
an individual report via Learning Central by the submission date and time (max 1000 words
and in size 11 Calibri font). You MUST use the template document provided on learning
central. You may include figures and graphs in your report: these DO NOT count towards
the page limit, but the total report length must not exceed 4 pages of A4. When submitting,
submit the file as a docx extension. This is important: do not submit your report in any
format (e.g., odt, pdf) other than docx. You must also submit a copy of the school cover
sheet as stated above.

Part I: Experiment Design and Conduct
You are tasked with designing an experiment and gathering data from human participants.
Pick ONE of the following problems:

• Design an experiment to assess the impact of keyboard and touch screen-based text
entry on text input speed.
• Design a study to compare the usability of TWO different ecommerce user interfaces
for browsing and choosing fashion products.
• Design a study to compare player experience between any TWO games of your
choice.

It is your responsibility to plan and conduct the experiment. For participant recruitment, you
will be told via an announcement on learning central and in lectures your recruit partner. At
a minimum, you should conduct your experiment with this person. You may choose to
recruit more participants if you like (just for fun….or perhaps to see how sample size may
impact your results!), but if you do so you must ONLY recruit participants from within
CMT206. You will also be told via an announcement on learning central and in lectures your
participate partner. At a minimum, you should participate in this person’s experiment. NB:
There are marks allocated for reflecting upon conducting your experiment and
participating in your partner’s experiment.

You must carefully plan out your study and choose appropriate research methods to apply.
Key considerations you must make are: 1) what variables to manipulate and measure, 2)
which instrument (e.g., questionnaire) is appropriate, 3) if a between or within groups
design is appropriate, and 4) what confounding factors might design decisions introduce and
how to mitigate them.

Part II: The Report
Write a reflective report detailing your contribution to the coursework. Your report should
be structured in 2 sections:

Section 1: My contribution (estimated 800 words)
In this section, you should detail your contribution to the experiment design,
critically reflecting on your contribution and the results your team obtained. At a
minimum, you MUST include the following: a clear description of the IVs and DVs in
your experiment and discussion about and a copy of the questionnaire you used in
your experiment. The questionnaire should be included as an appendix and does not
count towards the word limit. The rest is up to you. For example, you may include
detail on how data was collected and stored e.g., using a spreadsheet, and any raw
data collected. The key here is your contribution: make sure it is clear to the reader
what you learned from the process of designing and conducting human based
experiments. Finally, you may conclude on which prototype was ‘best’ with
reference to your results and report any changes you would make to your prototype
now that you have the results of your experiment.

Section 2: My participation (estimated 200 words)
In this section, you should reflect on your experience of participating in another
groups experiment. Think about how your experience as a participant compared to
that of a researcher and reflect on whether your appreciation for the coursework
differed from this perspective. For example, you may describe how the team briefed
and debriefed you for participation, or whether they had made different experiment
design decisions than your team and suggest reasons why they may have done so.

The word counts for each section are suggestions: you are free to divide the maximum word
count between the sections however you see fit. However, you should use the criteria for
assessment (see below) as a set of guidelines for your writing.

Learning Outcomes Assessed
5. Understand experimental design for the subjective assessment of user experience
6. Select and apply suitable methodologies for the conduct and analysis of a subjective
experiment.

Criteria for assessment
Section 1 is weighted at 0.75 of the total marks for the assignment while Section 2 is
weighted at 0.25 of the total marks.

For example, if you are awarded 3 marks from Section 1 and 8 marks from Section 2 then
your total percentage score would be: ((0.75 * 3) + (0.25 * 8)) * 10 = 42.5%. As another
example, if you are awarded 8 marks for Section 1 and 3 marks for Section 2 then your total
percentage score would be: ((0.75 * 8) + (0.25 * 3)) * 10 = 67.5%.


The marking scheme is as follows:
• Section 1:
o A clear description of the experiment, including the IVs, DVs, and
questionnaire: (4 out of 10 marks).
o Reflective reporting on the experiment design and conduct, justifying all
decisions made by reasoned argument and/or evidence: (6 out of 10 marks).
• Section 2:
o Descriptive information around the experiment you participated in. Note
there is no need to understand the design decisions in detail. Simply give an
account of what you did: (5 out of 10 marks).
o Reflective reporting demonstrating appreciation of the experience: (5 out of
10 marks).
Marks are awarded using the following criteria:

Level Fail (0-49%) Pass (50-59%) Merit (60-69%) Distinction (70-
100%)
Section 1 No/poor
description of the
experiment; no
description of the
IVs and DVs used;

No/poor
reflection on the
experiment
design; no/poor
justification of
decisions made;
no/poor
evidence
provided;
A description of
the experiment is
provided but it is
unclear and/or
difficult to
follow;

Some attempt
made at a
reflective report,
providing some
detail, but
decisions are not
justified by
reasoned
argument and/or
appropriate
evidence;
A clear and easy
to follow
description of the
experiment is
provided;

The reflective
report clearly
documents the
learning
experience,
showing good
self-criticism and
awareness of the
experiment
design’s
limitations;

Decisions are
justified using
good
argumentation
and/or
appropriate
evidence;

A clear and easy
to follow
description of the
experiment is
provided;

The reflective
report is
thorough,
documents the
learning
experience in
detail
highlighting how
challenges were
overcome, and
demonstrates
good self-
criticism of the
experiment
design’s
limitations;

Decisions are
justified using
strong
argumentation
and/or with
appropriate
evidence, and a
high standard of
writing is
maintained
throughout.
Section 2 No/poor
description of
participation;

No/poor
reflection
showing a lack of
appreciation of
the experience;
A description of
participation is
provided but it is
unclear and/or
difficult to
follow;

Some reflection
on the
experience is
provided but is
shallow and
shows a lack of
appreciation of
the experience;
A clear and easy
to follow
description of
participation is
provided;

The reflective
report shows
good
appreciation of
the experience;
A clear and easy
to follow
description of
participation is
provided;

The reflective
report is
thorough,
discusses how
the experience
has impacted the
learning
experience, and
demonstrates a
strong
appreciation of
the experience;


Feedback and suggestion for future learning

Feedback on your coursework will address the above criteria and will be returned to you via
Learning Central.

Feedback from this assignment will be useful for your dissertation.


Q3

You are asked to conduct a statistical analysis of a dataset resulted from a user study. You
are asked to present your study and findings in the form of a report.

Assessment
type
Max. mark Max. length Format Deadline
Report 20 2 Pages .docx 28 May 2021 at
9:30am

Your report will be assessed according to how well you are able to
• describe the dataset and research objectives, (4 marks)
• analyse the data and interpret the results of data analysis, and (8 marks)
• visualise the results of data analysis. (8 marks)

You should carry out a statistical analysis of the dataset to determine what the data tells
you about the user study and should visualise your results of data analysis in a way that
allows a reader to understand the findings of your study. You are free to use any software
package to conduct you study and generate visualisations.

1. The dataset
A user study was conducted to investigate the difference in display quality between two
different smartphones (i.e., referred to as S1 and S2). Also, the impact of gender on the
comparison between these two smartphones should be investigated. A number of
participants were recruited to assess the smartphones. Display quality was measured with
questionnaires, using the following method: each participant was asked to rate some
images displayed on S1 and some images displayed on S2, using a scale ranged from 0 (poor
quality) to 100 (excellent quality). The dataset is available on Learning Central as a Microsoft
Excel file. You should base your study entirely on the dataset provided.

2. The report
The report is limited to a maximum of TWO A4 pages including figures and tables. Arial font
size 11, with justification, should be used with Single line spacing throughout.

IMPORTANT! Any deviation from the instructions above will result in a deduction of 10% of
the maximum mark obtainable for the assessment. If you submit more than two pages as
instructed above, then ONLY THE FIRST TWO PAGES of the submission will be marked.
Extra submissions will be ignored.

IMPORTANT! Submission is via Learning Central. Submit your report as a single .docx file –
NO other attachments or files should be included.

You should write a clear and concise report. Your writing should be brief and to the point. A
suggested structure for your report is shown below.
• Introduction: should provide a context for the work, and a brief description of the
dataset and research aims.
• Data analysis: should consist of one or more sub-sections on a statistical analysis of
the data. You should formulate your research questions, describe the methods and
procedure of statistical tests and discuss the results of these tests. The titles of these
sub-sections should reflect what the sections contain.
• Data visualisation: should consist of one or more instances on the results of the data
analysis. You should include accurate and informative axis labels and titles. The
captions should provide sufficient information to the readers without referring to
the related text in the report.
• [References are not required for the report.]

Learning Outcomes Assessed

2. Evaluate competing proposals for interface design and implementation.
4. Demonstrate competency in the method of scientific analysis, the control of variables,
analysis, and the presentation of outcomes.
6. Select and apply suitable methodologies for the conduct and analysis of a subjective
experiment.

Criteria for assessment

Credit will be awarded against the following criteria.

Level Fail (0-49%) Pass (50-59%) Merit (60-69%) Distinction (70-
100%)
Description of
dataset and
research aims (4
marks)
No/poor
description of
context of
dataset.
No/poor
description of
research aims.
Context of
dataset and
research aims
described well.
Narrative is
sometimes
unclear.
Context of
dataset and
research aims
documented
well.
Narrative is clear
and coherent.

Context of
dataset and
research aims
thoroughly (with
cogent
justification)
documented.
Narrative is clear,
coherent, and
refined. A high
standard of
writing is
maintained
throughout.
Data analysis and
interpretation of
results (8 marks)
No/basic analysis
of data.
Data analysis
does not produce
meaningful
solutions for the
research aims.
Statistical
methods applied
correctly for the
most part.
Results
interpreted
correctly for the
most part.
Statistical
methods applied
correctly.
Results
interpreted
correctly.
Statistical
methods applied
carefully and
precisely (with
cogent
justification).
Results
interpreted
carefully and
precisely.
Data visualisation
and presentation
(8 marks)
No/poor
visualisation of
results.
No/poor
presentation of
story/findings.
Visualisation(s) of
results mostly
appropriate and
correct.
Presentation of
story/findings
mostly clear to
readers.
Visualisation(s) of
results
appropriate and
correct.
Presentation of
story/findings
clear to readers.
Visualisation(s) of
results carefully
and precisely
(with cogent
justification)
produced.
Presentation of
story/findings
clear and
accessible to
readers.

Feedback and suggestion for future learning

Feedback on your coursework will address the above criteria. Individual feedback and marks
will be returned on 25 June 2021 via Learning Central.

Feedback from this assignment will be useful for your dissertation.




欢迎咨询51作业君