Behind the Feed Data Challenge

Phase 1 – Data Analysis

Dates: Sept 14-Oct 18

Use publicly available datasets and other appropriate data sources to investigate patterns in online information exposure and recommendation systems. 

Your analysis could explore: 

• How different users or communities are exposed to different types of information 

• Whether certain topics, perspectives, sources or geographic areas are over- or underrepresented 

• How personalization may affect the diversity of information users encounter 

• Potential indicators of bias or unequal outcomes in recommendation systems 

• How fairness and diversity in information exposure can be measured 

• What limitations or biases exist within the data itself 

 

Teams may use recommended datasets such as news recommendation data, media and geographic datasets, movie or content recommendation datasets, or another relevant public dataset of their choice. 

Your deliverable: A data-driven visual summary that highlights patterns, disparities and insights related to information exposure and recommendation systems. Present your findings as a graphic or series of graphics (5 pages maximum, saved in PDF format). Your submission should tell a clear visual story using the data you have analyzed and clearly identify your data sources, methodology and limitations. All submissions must be in English and uploaded to the platform by 11:59 PM (ET) on October 18. 

Scoring: A panel of judges will score your submission based on the evaluation criteria between October 19-21. Teams will receive their scores on the evening of October 21. The top 20 teams will advance to Phase 2.

 

Phase 2 – Solution Development 

Dates: Oct 21-Nov 15 

Based on your findings from Phase 1, develop a data-informed solution, strategy, framework or tool that addresses an issue related to information exposure, algorithmic bias or fairness in recommendation systems. 

Your solution should aim to: 

• Improve diversity and fairness in information exposure 

• Increase transparency around how recommendation systems shape what people see 

• Identify or mitigate potential sources of algorithmic bias 

• Help users, organizations or technology developers better understand information exposure patterns 

• Support more equitable access to diverse perspectives, information and opportunities 

 

Your solution could take many forms. Examples include a bias audit framework, fairness metrics for recommendation systems, interactive dashboard, recommendation model, diversity-injection strategy, policy proposal, user-facing tool or other creative concept. 

During this phase, your team will have the option to meet with a mentor to support you in developing your idea. Mentors are experts in their field who will help you explore different aspects of your solution and strengthen your final submission. 

Your deliverable: A prototype, framework, policy proposal, strategy or other creative concept that addresses an issue identified through your Phase 1 analysis. Your solution should be grounded in your data findings and explain how it could contribute to a fairer, more transparent, or more diverse information environment. Submit your solution in a PDF document no longer than 5 pages in length. All submissions must be in English and uploaded to the platform by 11:59 PM (ET) on November 15. 

Scoring: Between November 16-18 a panel of judges will score your submission based on the evaluation criteria. Scores from Phases 1 and 2 will be combined to determine the top 5 scoring teams. The top 5 teams will advance to Phase 3. Results will be shared on the evening of November 18. 

 

Phase 3 – The Final Pitch  

Date: November 23 - Virtual Event 

 

Between November 18-23, the top 5 teams will refine their solution and prepare a presentation. Finalist teams will pitch their solutions to the jury at a virtual event on November 23 at 4:00 PM ET. 

Your deliverable: A live presentation about your solution (maximum 5 minutes), including: 

• The problem or disparity identified through your Phase 1 analysis 

• Key findings from your data analysis 

• Your proposed solution 

• How your solution could address the issue identified 

• Any limitations, trade-offs or considerations associated with your approach 

Judges may ask questions at the end of your presentation. 

Scoring: A panel of judges will score presentations based on the evaluation criteria. The top 3 teams will be determined through these scores and announced at the end of the event.

 

Data Starting Points

Teams may use publicly available datasets, APIs or other appropriately sourced data.

 

The following resources are recommended starting points: 

• Microsoft MIND News Recommendation Dataset: News articles and user behaviour data designed for recommendation systems research.  

• GDELT: Global news and media data that can be used to explore differences in topic, geographic and source representation. 

• MovieLens: User ratings and movie metadata that can be used to build and evaluate recommendation systems and investigate personalization and content diversity. 

• Other public datasets: Teams may identify their own dataset related to recommendation, information exposure, media representation or algorithmic fairness, provided that the data source and methodology are clearly documented. 

 

Teams are encouraged to consider whether their dataset contains sufficient information to support the conclusions they want to draw. Students should not infer demographic bias when demographic information is not present in the data. 

 

Every project should identify: 

1. Where the data came from 
2. What the data represents and does not represent 
3. What proxy they are using for "information exposure" 
4. What limitations or biases exist in the dataset 
5. Why their chosen fairness metric or comparison is appropriate