MARK3088
Product Analytics
Term 2, 2024
Assessment 1: Quizzes
Description of assessment task
ONLINE QUIZ 1 (10%)
Description: This is an online multiple-choice test, consisting of 10 questions covering week 1-3
Timing: The Online Quiz will be available on Week 3 Tuesday 5pm, and you must finalize your attempts to complete it by 11.59pm on 16th June Sunday. The quiz will be disabled after this time so make sure that you have completed it before then. My advice is to complete it a few days early to avoid any glitches that may occur.
ONLINE QUIZ 2 (20%)
Description: This is an online multiple-choice test, consisting of 20 questions covering week 1-5.
Timing: The Online Quiz in Week 5 will be available on 25th June Tuesday, and you must finalize your attempts to complete it by 11.59pm on 30th June Sunday. The quiz will be disabled after this time so make sure that you have completed it before then. My advice is to complete it a few days early to avoid any glitches that may occur.
Approach to the assessment task
Read the questions carefully and select the correct answer. You may attempt this quiz as many times as you choose (on each attempt questions will be sampled from a bank in a randomized order). The highest mark will be recorded. You will receive your mark and right answers right after the quiz closes.
Structure
This assessment task constitutes 30% of your overall mark for this course. Each question is worth 1 mark. You will receive 1 mark for answering a question correctly. Feedback and results will be provided through the Moodle quiz activity once the quiz window closes and after you have attempted the quiz.
Submission instructions
Remember you can take the quizzes as many times as you like until the quiz closing date. The highest mark will be recorded as the final mark.
Assessment 2: Case Study Analysis
Description of assessment task
Each student will write a report to investigate a business problem about new product development in the digital space. The case will enhance students' analytical, qualitative, and strategic decision-making skills as well as their knowledge of multidisciplinary marketing approaches.
Specifically, limit your response per question to a maximum of 2 A4 pages. In total, no more than 6 pages. For each specific task conducted, interpret the results and transform. the results into words (findings) i.e. based on your results, what further information and recommendations (e.g. managerial actions) you can provide to Colgate or LDC.
Approach to the assessment task
1. Sentiment analysis
Description of the dataset
With total 1000 rows, the dataset consists of 500 positive and 500 negative rows Colgate product reviews. All the rows have been generated through prompts given to ChatGPT-4. There is no redundancy/duplication in the rows since GPT generated only max 50 rows in one-go. The rows were carefully inspected to remove any duplicate found at run-time. This dataset can be used for basic sentimental analysis as this dataset mirrors real-world customer product reviews.
Here, sentiments on toothpaste reviews are collected and stored as .csv file, consisting of 2 attributes:
• Review: review of Colgate toothpaste.
• Sentiment: sentiment related to the review, i.e., positive or negative.
Instructions of this analysis
Perform. the following tasks:
a) Train a natural language model from scratch (you can choose the complexity of your model) that predicts the sentiment score (positive or negative) of a given review. To perform. this task, the dataset needs to be split a priori in a 80-20 train-validation dataset, respectively, at random. The inputs of your natural language models can be either “Document embeddings” or “Bag of words”, depending on your personal preferences. Remark: To perform. this task, it can be that you need to perform. some pre-processing steps.
b) Evaluate the performance of your trained model by calculating the precision, recall and F1-score measurements on the validation dataset.
c) Predict sentiment from text. Visualise the Sentiment Analysis using either Scatter Plots or Heat Map (your choice of method and variables).
2. Topic modelling
Description of the dataset
Product reviews are important for brands to not only develop new product features, but also continuous improvement to achieve product superiority over their competitors, command higher market shares and a premium price relative to their competitors. Here, a corpus of toothpaste product reviews published in a British supermarket in the United Kingdom. It is stored as .csv file, consisting of 6 attributes:
• Rating: between 1 to 5
• ReviewText: Text of the toothpaste review in English
• SubmissionTime: Date of publishing for the product review in YYYY-MM-DDTHH:MM:SS.000+00:00 format
• Title: Text title of each review in English
• UserLocation: Location of the reviewer in city, region, country format
• UserNickName: Reviewer's nickname
Instructions of this analysis
Perform. the following tasks:
a) Apply the Latent Semantic Analysis and Latent Dirichlet Allocation technique to study the topic focus of toothpaste’s reviews. Characterize them by exploring the most frequent words in each topic.
b) How do these topics evolve through time in the product review?
Remark: To perform. this task, it can be that you need to perform. some pre-processing steps.
3. A/B Test
Description of the dataset
Reviewing the LDC case study again together with both Week 1 and Week 3 lecture slides as well as Week 4 tutorial slides. LDC tested its product at each stage of the customer journey. LDC has implemented experimental design for InstaMoney. The meaning of each variable is explained through the spreadsheet called “Variable Legends”.
Instructions of this analysis
Perform. InstaMoney Prototype A/B testing results (both conversion and lift) for the following features:
a) Generating Prospects
b) Onboarding
c) Credit Assessment
d) Payment Reminder
e) Query Resolution .
Based on the A/B testing results, how can LDC incorporate the best feature into InstaMoney?
Submission instructions
• Questions 1 and 2 are designed for Orange, consisting of a clear workflow so that the steps and conclusions you make can be easily followed and verified.
• Question 3 is designed to be answered using excel although Orange also allows you to select the plotting method in Scoring method to conduct statistical analysis, such as t-test and ANOVA. Both tools are acceptable. The key is to draw conclusion to ensure the best product features.
• Create a report using Microsoft Word, including cover page, index, headings, and visuals.
• Screenshot of your completed workflow from Orange.
• Include in your report answers to the questions 1-3 outlined above. Limit your response per question to a a maximum of 2 A4 pages (single space, 12 fonts, 2.5cm margins, Calibri). Hence, a total number of 6 pages can be submitted.
• Always provide rational to the insights you have gathered.
• Feel free to customize the report based on specific findings and nuances from your analysis.
• N Name your report as Tutorial_PA_Orange_zID.doc (or .docx). For example: W14A_PA_Orange_z1234567.doc.
• Name your Orange workflow as Tutorial_PA_Orange_QuestionID_zID.ows. For example: W14A_PA_Orange_Q1_z1234567.ows. OR W14A_PA_Orange_Q2_z1234567.ows.
• For Q3 and excel is chosen, name your file as W14A_PA_Excel_Q3_z1234567.xls (or .xlsx).
• This assignment contains four submission parts: report, workflow of question 1, workflow of question 2, workflow or excel of question 3. Submit the files via the corresponding upload links on Moodle by 14th July Sunday 11:59 pm.
• Late submission results in a 5% penalty for every day late, with a maximum of 5 days, impacting the grade negatively.
Assessment 3: Group Project
Description of assessment task
You will develop a new data product with your group members. This project provides you with an opportunity to take your knowledge and skills of the product analytics learned in the course and apply them to a real-world problem. The purpose of group presentation is to communicate what you are building, who it is for, and how it will deliver value to end users and to the business.
Approach to the assessment task
This assessment provides you with the opportunity to apply the concepts learned in class to develop and communicate new data products. Your group will identify customers’ pain points and unmet needs in each stage of the customer journey and suggest potential solutions. Then, your group will plan a new product measurement framework, develop a tracking plan, and test your product. Finally, your group will communicate your work using video recording. In classes, all of us will watch the recorded videos and then discuss them together.
Milestones, scaffolding, informal feedback opportunities during tutorials.
• Week 3: Group formation with each group of 4-5 students. Elect the roles of CEO (group leader), developer, product manager, product marketing manager, UX designer. Confirm the app product or app service. (Keep a diary re: teamwork challenges, solutions and skills developed)
• Week 4: Part 1 Creating “Customer Journey Map”
• Week 8: Part 2 Creating “Measurement Framework”
• Week 9: Part 3 Creating “Tracking Plan”
Week 10: Presentation
Submission instructions
As you are aware now, there are a number of delivery dates for this project. Although they are formative coursework which does NOT count towards the final mark, completing the tasks in the respective week helps to progress your group project. Make sure you work with your group and individually to format and complete the delivery following these requirements:
• Always provide rational to the insights you have gathered and use academic support to strengthen your arguments.
• For presentations, make sure you work on a clear and professional presentation. Remember that you do not need to include all the materials you have gathered. You will need to make a group decision to organise the most important materials.
• As part of your slide presentation, please add speaker notes to each slide. These notes are important because they provide context, explanations, and additional details that may not be evident from the slide content alone. The notes help you convey your message effectively to the audience and ensure a comprehensive understanding of your presentation.
• All group members must be involved in the presentation.
• Name your presentation as Tutorial_PA_zID-groupleader.ppt (or .pptx). For example: T11A_PA_z1234567.ppt. Note that only ONE member of the group submits the deck of slides via Turnitin.
• Submit the file via an upload link on Moodle by 29th July Monday noon 12:00 pm. Late submission results in a 5% penalty for every day late, with a maximum of 5 days, impacting the grade negatively.