# Step 8: Challenge

Step 8 - Challenge

{% hint style="info" %} This step will require you to download the new learning set that we sent you (*MEDomicsLab\_TestingPhase\_Step8.zip*). This set comprises two datasets, combining the learning and holdout sets obtained in[ *Step 4 - Explore Data*](/medomics-docs/medomicslab-docs-v0/test-with-mimic/step-4.md) at *Time Point 1* and *Time Point 2*. To avoid confusion among all the datasets from the beginning of the Testing Phase, we recommend creating a new workspace for this step containing only the new learning set. An invitation to access the *MEDomicsLab\_TestingPhase\_Step8.zip* data has been sent via email. {% endhint %} Welcome to the final step of the MEDomicsLab Testing Phase! We appreciate your engagement throughout this journey. In *Step 8 - Challenge*, you will leverage the knowledge you have gained from the MEDomicsLab platform in the preceding steps of the Testing Phase. You are the master here :clap:! The objective here is that participants design their own model via the use of the [*Machine Learning (ML) Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/development/learning-module)*.* These models will be evaluated by the MEDomicsLab team using a fresh, private holdout set from the [MIMIC](https://mimic.mit.edu/) database. {% hint style="info" %} Considering the insights we have gained during the Testing Phase, we have now updated the rules for this step. In contrast to what we initially planned, there will be a single challenge that involves manipulations in the [*Machine Learning (ML) Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/development/learning-module) only. {% endhint %} {% hint style="info" %} We are continuously working on enhancing the MEDomicsLab platform, and we would like to inform you that there may still be some missing options and tooltips in the [*Machine Learning (ML) Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/development/learning-module), which we intend to implement in the future. {% endhint %} ## Recommendations Before proceeding with *Step 8 - Challenge* of the MEDomicsLab Testing Phase, we recommend revisiting the documentation related to [*Step 6 - Create Model*](https://medomics-udes.gitbook.io/medomicslab-docs/test-with-mimic/step-6). {% content-ref url="/pages/ulhe5SbRPDOsL07NE0o5" %} [Step 6: Create Model](/medomics-docs/medomicslab-docs-v0/test-with-mimic/step-6.md) {% endcontent-ref %} ## Instructions for Step 8 - Challenge * Download the new learning set provided in *MEDomicsLab\_TestingPhase\_Step8.zip*. * Create a model similarly to the one we created in [*Step 6 - Create Model*](https://medomics-udes.gitbook.io/medomicslab-docs/test-with-mimic/step-6) using the [*Machine Learning (ML) Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/development/learning-module). Follow these guidelines for creating your best possible model: * Train, optimize, and test models. * Compare different data combinations. * Choose a learning hypothesis and create a final model. * Once you have trained your best model, send it to with "Step 8 - Challenge | Submission" as the Subject of your email. You are allowed to send multiple submissions, but please note that we will only evaluate your latest submission. * If you wish, you can let us know in the email the name that we should use for the public ranking of your submission (e.g. *MLrocks* :smile:). If not specified, we will use your full name. {% hint style="warning" %} For this *ML Challenge*, most of your work will therefore be done inside the [*Machine Learning (ML) Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/development/learning-module). However, you are also welcome to use the other capabilities of the MEDomicsLab platform. For example, you may want to use the [*Exploratory Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/design/exploratory-module) to continue to gain useful insights about the dataset prior to the learning step. However, to facilitate the Evaluation process of the models submitted by the participants, **we please ask the following:** * ***Only use*** the data we provided in the *MEDomicsLab\_TestingPhase\_Step8.zip file.* * Therefore, you should not be able to use the *Extraction Module* or the *MEDprofiles* package*.* * ***Do not change*** column names in the data we provided (this includes the use of the Groupping/Tagging tool). * ***Do not merge*** the time point CSV files together. * ***Dot not use*** the "Transform Columns tool". * ***Do not use*** the "PCA" utility of the "Feature Reduction tool". * If you use the "Spearman" utility of the "Feature Reduction tool", **make sure to keep the *****subject\_id***** and *****target***** columns in your dataset** by enabling the "Merge unselected columns in the result dataset" and "Keep target in dataset" options. * Note that you new dataset will be placed in the *reduced\_features* folder and that you will have to move it in your *learning* folder to retrieve it in the [*Learning Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/development/learning-module). If these instructions are not followed, we will not be able to evaluate your submission, and it will unfortunately be discarded. Also, note that to create a model in the [*Learning Module*](https://medomics-udes.gitbook.io/medomicslab-docs/tutorials/development/learning-module), your CSV files must be placed under a "learning" folder and have the prefix "TX\_" (e.g., *T1\_new\_learning*) as in the data we provided you. {% endhint %} {% embed url="" %} **Content** Intro [0:00](https://www.youtube.com/watch?v=AMiNfv1J67g\&t=0s) Rules [0:38](https://www.youtube.com/watch?v=AMiNfv1J67g\&t=38s) Dealing with Server Errors [4:03](https://www.youtube.com/watch?v=AMiNfv1J67g\&t=243s) Documentation [5:08](https://www.youtube.com/watch?v=AMiNfv1J67g\&t=308s) The Optimize node [8:58](https://www.youtube.com/watch?v=AMiNfv1J67g\&t=538s) General Advice [11:05](https://www.youtube.com/watch?v=AMiNfv1J67g\&t=665s) *** {% hint style="info" %} After the conclusion of the Testing Phase on May 13, we will evaluate the performance of the models submitted by all participants and establish a ranking through an evaluation using our private holdout set. Following this, we will make the ranking public and reveal the winners of both the *ML Challenge* :tada: and the *Bug Finder Challenge* :smile: at the **wrap-up meeting scheduled for May 17**. An invitation to this meeting will be sent out soon. {% endhint %} --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://medomicslab.gitbook.io/medomics-docs/medomicslab-docs-v0/test-with-mimic/step-8.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.