Page 1

OENG1116 – Summative Assessment 2

Individual Project Portfolio

1 Assignment context

This project for OENG1116 is based on your work developing and selecting the most appropriate model for

an engineering system, interpreting the simulation results of that model and comparing the results of different

model architectures. This assessment task builds on project by using lecture, tutorial classes, activities

undertaken during the semester till week 12 (Submission date is Friday, 5 June 2020, Time:

23.59 ). The following documents can be used for completion of this assessment

• Lecture notes

• Tutorial notes

• Matlab toolbox (as indicated below)

• Book chapters:

(i)H. Khayyam, G. Golkarnarenji, R.N. Jazar, “Limited Data Modelling Approaches for Engineering

Applications”, In Nonlinear Approaches in Engineering Applications; Jazar, R.N., Ed.; International

Publication Springer: Cham, Switzerland, (2018)

(ii) B Crawford, H Khayyam, AS Milani, RN Jazar, “Big Data Modeling Approaches for Engineering

Applications” Nonlinear Approaches in Engineering Applications, 307-365 (2019)

2 Assignment overview.

This individual assessment requires the student to present a project report, based on the activities proposed.

The overall aim of this assignment report is to demonstrate the technical and non-technical learning

outcomes of the unit by the student. This assignment has an overall weight of 35 % of the course.

Page 2

3 Learning Outcomes

A summary of the Course Learning Outcomes (CLOs) which will be assessed in this task are provided in table

1.

Table 1: Summary of CLOs assessed in Assessment 2.

This task assesses your

Course Learning

Outcomes (CLOs)

CLOs

1- Analysis

Ability to model non-deterministic (heuristic) systems and differentiate

between nonlinear and linear models.

Ability to numerically simulate linear and non-linear deterministic systems.

Ability to estimate and validate a model based upon input and output data.

Ability to create a model prediction based upon new input and validate the

output data.

Ability to understand and apply advanced theory of engineering fundamentals

and specialist bodies of knowledge in the selected discipline area to predict

the effect of engineering activities.

Ability to apply underpinning natural, physical and engineering sciences,

mathematics, statistics, computer and information sciences to engineering

applications.

2- Research

Ability to plan and execute a substantial research-based assessment tasks, with

creativity and initiative in new situations in professional practice and with a

high level of personal autonomy and accountability.

Awareness of knowledge development and research directions within the

engineering discipline.

Ability to develop creative and innovative solutions to (heuristic) engineering

challenges.

Ability to assess, acquire and apply the competencies and resources

appropriate to engineering activities.

Ability to demonstrate professional use and management of information.

Ability to clearly acknowledge your own contributions and the contributions

from others and distinguish contributions you may have made as a result of

discussions or collaboration with other people.

Page 3

4 Assignment details and requirements

Your report is related to the development of three different models for the given experimental data

shown in Tables 2 and 3. The aim of the report is to justify all the decisions that you made to develop

the different models, showing your skills to analyse non-deterministic (heuristic) systems.

Table 2: Experimental data (Training)

No.

Inputs Outputs

Input1 Input2 Input3 Output1

1 227 20 1 1.2446

2 227 20 4 1.2438

3 227 25 2 1.25

4 227 25 3 1.2417

5 227 30 3 1.2359

6 227 35 4 1.2244

7 230 20 2 1.2574

8 230 25 1 1.2417

9 230 25 3 1.2464

10 230 30 4 1.2341

11 230 35 1 1.2335

12 230 35 3 1.2317

13 233 20 3 1.257

14 233 25 1 1.2611

15 233 20 4 1.2601

16 233 25 2 1.2457

17 233 25 3 1.2465

18 233 25 4 1.2565

19 233 30 1 1.2429

20 233 30 3 1.2421

21 233 35 2 1.2363

22 236 20 1 1.2707

23 236 20 4 1.271

24 236 25 3 1.263

25 236 30 2 1.2547

26 236 35 1 1.2504

27 236 35 4 1.2474

Page 4

Table 3: Experimental data (Testing)

No.

Inputs Outputs

Input1 Input2 Input3 Output1

28 227 35 1 1.2339

29 230 20 4 1.2588

30 233 35 3 1.2372

The output required for this assessment task is based on the 4 key areas as defined in Table 4, which

provides full descriptions of the functionalities required for each model. In addition, the approximate

length of the content has been specified (though not fixed) including the weight of each area

(Considering the total 35 % of the assignment).

Table 4: Description of key tasks required for report.

Item Task Name (output) Description ULO(s)

Approx.

Length Weight

1

Modelling

using

Artificial

Neural

Network

Read the collected data from Table 2. Perform data pre- processing if

required. Develop a predictive model of input-output data sets based

on Artificial Neural Networks (ANN) in MATLAB. Split the data

into relevant ratios for training, validation and testing, providing

justification on the ratios chosen.

(i) Describe the network architecture, training procedure and every

step carried out to improve the model.

(ii) Define the fitting neural network through changing the number of

hidden neurons (for example 5-25).

(iii) If applicable, find a solution to achieve the best fit in terms of

model performance by choosing and controlling different training

algorithms (Levenberg–Marquardt, Conjugate Gradient, Quasi-

Newton Algorithms, Bayesian Regularization, Gradient Decent).

(iv) Evaluate the accuracy (error) of the developed model by using

the data provided in Table 3.

CLO1 ~2 to 3 page 10%

2

Modelling

using

Support Vector

Machine

Read the collected data from Table 2. Perform data pre-processing if

required. Develop a predictive model of input-output data sets based

on different Support Vector Machine (SVM) in MATLAB.

(i) Describe the training procedure and every step carried out to

improve the model.

(ii) If applicable, find a solution to achieve the best fit in terms of

model performance (use different SVM kernels Linear, Gaussian and

Polynomial).

(iii) Evaluate the accuracy (error) of the developed model by using the

data provided in Table 3.

CLO1 ~ 2-3 pages 10%

3

Modelling

using

Linear Non-

Linear

Regression

Read the collected data from Table 2. Perform data pre-processing if

required. Develop a predictive model of input-output data sets based

on different Non-Linear Regression (NLR) in MATLAB.

(i) Describe the training procedure and every step carried out to

improve the model.

(ii) If applicable, find a solution to achieve the best fit in terms of

model performance (use different NLR models: Polynomial,

Exponential, Power and combination).

(iii) Evaluate the accuracy (error) of the model using the test dataset

on Table 3.

CLO1 ~ 2-3 pages 10%

Page 5

4

Find

RMSE, MSE

and R

Find MSE, RMSE and R for the models (1-3).

Note: Use equations 1-3 to calculate RMSE, MSE and R:

CLO1 ~ 1 page 2.5%

5

Compare

Compare the three methods used. Discuss on the

advantages/disadvantages of the different models in this application CLO1 ~ 1 page 2.5%

Notes on structure and formatting: To make this task as simple as possible, the structure of the report

should be based exactly on the tasks defined above. That is, you should have 7 sections (5 tasks and 2

Appendixes) in your report which contain the headings defined by the 5 Tasks Name in Table 4, Appendix A:

Different ANN, SVM and NLR Methods Results and Appendix B: ANN, SVM and NLR Matlab Codes.

There is no need for additional introduction and conclusion sections, or formatting such as Table of Contents,

List of Figures, etc. However, you will still be assessed on the quality of the report and the clarity of the

communication, via the assessment of CLO1- 3 and throughout the report.

5 Marking criteria

The assessment criteria are based on how well you have completed the 4 tasks defined in Table 4.

• You will be scored for each of the key tasks defined in Table 4. The marks will then be weighted

according to the marking rubric shown in Table 5.

• To achieve the maximum score for each task, you will have clearly covered the information provided in

the description, demonstrating that you have met the relevant Course Learning outcomes defined for each

of the tasks in Table 4.

Page 6

Table 5. Project Assessment Rubric

Topic (Mark) Unsatisfactory

(0<30%)

Satisfactory

(30%<70)

Good

(70%<80%)

Outstanding

(>80%)

Task 1 Errors in reading the input/output

data. No attempt on pre- processing

data or no justification. Develop a

predictive model of input- output

data sets based on Artificial Neural

Networks (ANN) in MATLAB. The

ratios chosen for training, validation

and testing are not justified. There is

little or no justification on the

parameters chosen to build the ANN

(i)Brief descriptions of the network

architecture, training procedure and

every step carried out to improve the

model.

(ii) Lack of mathematical analysis of

the results

(iii) Code shows errors, does not

work properly or there is no code

shown for the ANN

Successfully reading the collected data from

Table 3, with proper justification. Perform

data pre- processing if required. Develop a

predictive model of input- output data sets

based on Artificial Neural Networks (ANN)

in MATLAB. Split the data into Different

ratios for training, validation, and testing,

discussing on the effects of different ratios

on the ANN performance.

(i)Describe the network architecture, training

procedure and every step carried out to

improve the model, showing a clear

understanding of the effects of changing the

different parameters considered

(ii) Define the fitting neural network through

changing the number of hidden neurons (for

example 5-25).

(iii)If applicable, find a solution to achieve

the best fit in terms of model performance

by choosing and controlling different

training algorithms (more than one)

(Levenberg–Marquardt, Conjugate Gradient,

Quasi-Newton Algorithms, Bayesian

Regularization, Gradient Decent).

Successfully reading the collected data from Table

3, with proper justification. Perform data pre-

processing if required. Develop a predictive model

of input- output data sets based on Artificial Neural

Networks (ANN) in MATLAB. Split the data into

Different ratios for training, validation, and testing,

discussing on the effects of different ratios on the

ANN performance.

(i)Describe the network architecture, training

procedure and every step carried out to improve the

model, showing a clear understanding of the effects

of changing the different parameters considered

(ii) Define the fitting neural network through

changing the number of hidden neurons (for

example 5-25).

(iii)If applicable, find a solution to achieve the best

fit in terms of model performance by choosing and

controlling different training algorithms (more than

one) (Levenberg–Marquardt, Conjugate Gradient,

Quasi-Newton Algorithms, Bayesian Regularization,

Gradient Decent).

(iv) Mathematical analysis of the different ANNs

performance provided to show less error.

Successfully reading the collected data from Table 3, with

proper justification. Perform data pre- processing if required.

Develop a predictive model of input- output data sets based on

Artificial Neural Networks (ANN) in MATLAB. Split the data

into Different ratios for training, validation, and testing,

discussing on the effects of different ratios on the ANN

performance.

(i)Describe the network architecture, training procedure and

every step carried out to improve the model, showing a clear

understanding of the effects of changing the different parameters

considered

(ii) Define the fitting neural network through changing the

number of hidden neurons (for example 5-25).

(iii)If applicable, find a solution to achieve the best fit in terms

of model performance by choosing and controlling different

training algorithms (more than one) (Levenberg–Marquardt,

Conjugate Gradient, Quasi-Newton Algorithms, Bayesian

Regularization, Gradient Decent).

(iv) Mathematical analysis of the different ANNs performance

provided to show less error.

(v) Code compiles and work properly, built with a logical

structure and including enough comments to help understanding

the ANN developed.

Task 2 Errors in reading the input/output

data. No attempt on pre- processing

data or no justification. Develop a

predictive model of input- output

data sets based on Support Vector

Machine (SVM) in MATLAB.

Input/output data is not used

correctly for a SVM

(i)Brief descriptions of the network

architecture and every step carried

out to improve the model.

(ii) Lack of mathematical analysis of

the results

(iii) Code shows errors, does not

work properly or there is no code

shown for the SVM

Successfully reading the input/output data.

Perform data pre- processing if required.

Develop a predictive model of input-output

data sets based on Support Vector Machine

(SVM) in MATLAB

(i) Detailed description of the model

architecture, and the steps carried out to

improve the model, with proper

justification of every parameter considered.

(ii)If applicable, find a solution to achieve

the best fit in terms of model performance

by choosing and controlling different

training algorithms (more than one)

((Linear, Gaussian and Polynomial).

Successfully reading the input/output data. Perform

data pre- processing if required. Develop a

predictive model of input-output data sets based on

Support Vector Machine (SVM) in MATLAB

(i) Detailed description of the model architecture,

and the steps carried out to improve the model, with

proper justification of every parameter considered.

(ii)If applicable, find a solution to achieve the best

fit in terms of model performance by choosing and

controlling different training algorithms (more than

one) ((Linear, Gaussian and Polynomial).

(iii) Evaluation of the accuracy of the different SVR

models. Mathematical analysis of the SVRs

performance provided to show less error.

Successfully reading the input/output data. Perform data pre-

processing if required. Develop a predictive model of input-

output data sets based on Support Vector Machine (SVM) in

MATLAB

(i) Detailed description of the model architecture, and the steps

carried out to improve the model, with proper justification of

every parameter considered.

(ii)If applicable, find a solution to achieve the best fit in terms of

model performance by choosing and controlling different

training algorithms (more than one) ((Linear, Gaussian and

Polynomial).

(iii) Evaluation of the accuracy of the different SVR models.

Mathematical analysis of the SVRs performance provided to

show less error.

iv) Code compiles and work properly, built with a logical

structure and including enough comments to help understanding

the SVR developed

Task3 Errors in reading the input/output

data. No attempt on pre- processing

data or no justification. Develop a

Successfully reading the input/output data.

Perform data pre- processing if required.

Develop a predictive model of input-output

Successfully reading the input/output data. Perform

data pre- processing if required. Develop a

predictive model of input-output data sets based on

Successfully reading the input/output data. Perform data pre-

processing if required. Develop a predictive model of input-

output data sets based on Non-Linear Regression (NLR) in

Page 7

predictive model of input-output data

sets based on Non-Linear Regression

(NLR) in MATLAB.

Input/output data is not used

correctly for a NLR

(i)Brief descriptions of the

implementation of the mathematical

model and every step carried out to

improve the model.

(ii) Lack of mathematical analysis of

the results

(iii) Code shows errors, does not

work properly or there is no code

shown for the NLR

data sets based on Non-Linear Regression

(NLR) in MATLAB.

(i) Detailed description of the model

architecture, and the steps carried out to

improve the model, with proper

justification of every parameter considered.

(ii) If applicable, find a solution to achieve

the best fit in terms of model performance

by choosing and controlling different NLR

approaches (more than one) (Polynomial,

Exponential, Power and combination).

Mathematical analysis of the different

NLR performances provided (to show the

error).

Non-Linear Regression (NLR) in MATLAB.

(i) Detailed description of the model architecture,

and the steps carried out to improve the model, with

proper justification of every parameter considered.

(ii) If applicable, find a solution to achieve the best

fit in terms of model performance by choosing and

controlling different NLR approaches (more than

one) (Polynomial, Exponential, Power and

combination). Mathematical analysis of the different

NLR performances provided (to show the error).

(iii) Evaluation of the accuracy of the different NLR

models. Mathematical analysis of the NLRs

performance provided

MATLAB.

(i) Detailed description of the model architecture, and the steps

carried out to improve the model, with proper justification of

every parameter considered.

(ii) If applicable, find a solution to achieve the best fit in terms of

model performance by choosing and controlling different NLR

approaches (more than one) (Polynomial, Exponential, Power and

combination). Mathematical analysis of the different NLR

performances provided (to show the error).

(iii) Evaluation of the accuracy of the different NLR models.

Mathematical analysis of the NLRs performance provided

(iv) Code compiles and work properly, built with a logical

structure and including enough comments to help understanding

the NLRs developed.

Task 4

No mathematical analysis, wrong

RMSE, MSE and R values

Only one of the parameters (RMSE, MSE or

R) is correct

Two of the parameters (RMSE, MSE or R) are

correct

Correct calculation of RMSE and MSE

and R

Task 5 No discussion of the findings or poor

discussion with lots of fundamental

mistakes.

Description of the findings with no

comparison/analysis between methods. The

description does not contain fundamental

mistakes.

Correct description of the findings and

comparison/analysis between the three different

models. Minor mistakes in the analysis.

Correct description of the findings and comparison/analysis

between the three different models. Discussion clearly shows a

strong understanding of the models built.

Page 8

6 Submission requirements

It is required to submit two files: a Report PDF file and a Matlab file on Canvas as follows:

1. Report PDF file: Prepare a report PDF file including the 5 tasks and Matlab code. PDF file are

strongly recommended. Other file formats will not be accepted. (Use the name_surname_ID and

extension for example: hamid_khayyam_s33784.PDF

2. Matlab file: Prepare a Matlab file including the 3 Matlab code with format m file. Other file formats

will not be accepted.

a. If you created a single Matlab file with different sections, name the file name_surname_ID.m

b. If you created a Matlab file for every different section name them as follow

i. name_surname_ID_ANN.m

ii. name_surname_ID_SVM.m

iii. name_surname_ID_NLR.m

Then zip all the Matlab files of them in a unique file with the named name_surname_ID.zip

3. To access Assignment, please, LOG IN into your RMIT account and then access the Canvas:

4. Upload both the PDF file and the Matlab code.

7 Penalties and bonus marks

Grace period for submission is 30 minutes. There are no bonus points associated with this assessment task