Take the challenge!

Register to compete in this year’s BIOMAG data challenge!

Competition information

Background

Cogitate is an international adversarial collaboration aimed at testing contrasting predictions of two neuroscientific theories of consciousness: Global Neuronal Workspace Theory (GNWT) and Integrated Information Theory (IIT). One of these predictions pertains to the interareal connectivity associated with the conscious experience. Specifically, GNWT predicts that a conscious experience requires phase-synchronization in the gamma/beta band between prefrontal cortex (PFC) and category-selective visual areas in the temporal lobe (i.e., FFA for faces and LOC for objects). IIT predicts phase-synchronization between these category-selective areas and low-level visual areas (e.g., V1/V2) (for more details, see Melloni et al. 2023)

To test these hypotheses, neuroimaging (fMRI) and neurophysiological data (MEG, EEG, and ECoG) were acquired from healthy volunteers and patients with pharmaco-resistant focal epilepsy while they were presented with sequences of suprathreshold images from four different categories (i.e., faces, objects, letters, and false fonts) and for three different durations (0.5s, 1s and 1.5s). Participants were asked to search for two infrequent targets prompted at the beginning of each sequence. Depending on the category of these targets, non-target stimuli could belong to the same or a different category, thus resulting in two distinct task-relevance conditions (see Melloni et al. 2023; Cogitate et al. 2023).

The Cogitate Connectivity Challenge will focus on the MEG data sets collected at the Centre for Human Brain Health (University of Birmingham, UK) and at the Centre for MRI Research (Peking University, China) using the TRIUX MEGIN system (for more details on data acquisition, see again (see Melloni et al. 2023; Cogitate et al. 2023).

Aim of the competition

So far, the Cogitate consortium has not been successful in establishing evidence for functional connectivity in MEG regarding gamma/beta band coherence as initially hypothesized. This evidently challenges the specific predictions of those two theories of consciousness, but it also highlights the more general question on whether gamma/beta band coherence is the ideal measure for establishing connectivity in task-based MEG data.

Specific aims

Make a case for meaningful measures of functional connectivity that can be applied to the Cogitate MEG dataset to test the predictions, i.e., connectivity that is selectively modulated by stimulus category (and task relevance).

Any measure is allowed (including phase coherence in the gamma/beta band).

Interpret the outcomes of the functional connectivity measures considering the specific predictions made by GNWT and IIT.


How to access the data

Data have been collected from ~100 participants. The MEG data for the initial 48 participants will be made available starting March 25, 2024. The complete dataset will be released on June 15, 2024

  1. Register as a Cogitate Data User here.

  2. Data will be released in two batches, and available to download by competition registrants who have created a Cogitate Data User account and thereby agreed to the terms of use and GDPR requirements.

  3. Login to your Cogitate Data User account here

  4. Download batch 1, MEG and Eye Tracking data bundles here | available after March 25

  5. Download batch 2, MEG and Eye Tracking data bundles here | available after June 15

For details on data acquisition parameters, task design, theories’ predictions, etc., see Melloni et al. 2023 and Cogitate et al. 2023.

Timeline

  • Batch 1 MEG data release: March 25, 2024

  • First (draft) report due (aka preregistration): June 14, 2024 [23:59 GMT]

  • Batch 2 MEG data release: June 15, 2024

  • Replication report due: July 31, 2024 [23:59 GMT]

  • Competition scoring: August 1 - August 25, 2024

  • Winner announced: @ BIOMAG 2024 in Sydney Australia, August 26-29, 2024

How to submit your entry

The teams are required to submit a written report supplemented by figures and codes (max 8 pages). The report should comprehensively address all the points outlined above. The results should ideally be reported in the format of Figure 4 in Cogitate et al. (2023). The provided code should be replication-friendly and with clear comments. At the end of the challenge, all submitted materials will be made public.

The deadline for first submission is June 14, 2024. We expect a draft of the report + code based on the first half of the participant data.

The deadline for second submission is July 31, 2024. The aim is for the teams to replicate their findings on the second half of the data (according to draft report, which will serve as a pre-registration).

Provide a statistical assessment of the connectivity results found with the adopted measures.

Make explicit how concerns on overfitting and reliability were addressed (e.g., consider developing measures on one part of the data set and test on the second part).

Address potential concerns complicating the interpretation (e.g., field spread and event-related fields).

Evaluation criteria and scoring system

1 | Relevance of the measure:

  • Demonstrate a clear rationale for the chosen functional connectivity measure.

  • Explain how the measure aligns with the aim of the challenge and with the specific predictions of GNWT and IIT

2 | Statistical Assessment:

  • Use appropriate statistical tests to evaluate the significance of the findings.

3 | Interpretation:

  • Clearly interpret the functional connectivity outcomes in the context of GNWT and IIT predictions

  • Discuss any unexpected findings and potential implications for the theories

4 | Concerns on Interpretation:

  • Address potential confounding factors such as field spread and event-related fields.

  • Discuss strategies employed to mitigate these concerns and their impact on the interpretation.

5 | Clarity and Quality:

  • Present a well-organized and clear report.

  • Include relevant figures that enhance the understanding of the results.

  • Include relevant codes to allow the replication of the analysis.

6 | Reliability and Replication:

  • Describe measures taken to address concerns of the reliability of the results.

  • Replicate findings on the second half of the dataset.

Prizes

20 points

20 points

20 points

15 points

10 points

15 points

Total Score: 100 points

  • €1000

  • €500

  • €250

Organizers

  • Oscar Ferrante

    Centre for Human Brain Health, University of Birmingham, United Kingdom

  • Ling Liu

    Peking University, Beijing China

  • Ole Jensen

    Centre for Human Brain Health, University of Birmingham, United Kingdom

  • Huan Luo

    Peking University, Beijing China

  • Praveen Sripad

    Max Planck Institute for Empirical Aesthetics, Germany

  • Tanya Brown

    Max Planck Institute for Empirical Aesthetics, Germany

  • Lucia Melloni

    Max Planck Institute for Empirical Aesthetics, Germany

  • Liad Mudrik

    Tel Aviv University, Israel

  • Niccolò Bonacchi

    Champalimaud Foundatio

  • Cogitate Consortium

Competition has now closed.

Thank you to everyone that participated and followed along!

GOOD LUCK TO ALL PARTICIPANTS

👍

GOOD LUCK TO ALL PARTICIPANTS 👍