FINAL REMINDER: UMAP Workshop on Comparative Evaluation in User Modeling, Adaptation and Personalization (EvalUMAP 2016)

Dear Community,

This is a final call for an exciting Workshop related to comparative 
Evaluation in UMAP (User Modelling and Personalization). It is also an 
excellent opportunity to connect, discuss and exchange experiences 
related to this important and challenging topic.

Please do not hesitate to reach out to me if you have any questions.

Hope to see you very soon in Halifax!
Kevin k.



On 06/04/2016 22:13, Kevin Koidl wrote:
>
> ======================== FINAL CALL FOR PAPERS =======================
>
> EvalUMAP 2016 Workshop
>
> Towards comparative evaluation in user modeling, adaptation and 
> personalization
>
> http://evalumap.adaptcentre.ie/
>
> ================================================================
>
> To be held in conjunction with the 24th Conference on User Modeling, 
> Adaptation and Personalization, UMAP 2016, July 2016, Halifax, Canada
>
> Call for Papers
> *Submission deadline: May 7, 2016*
>
> Research in the areas of User Modelling, Adaptation and 
> Personalization faces a number of significant scientific challenges. 
> One of the most significant of these challenges is the issue of 
> comparative evaluation. It has always been difficult to rigorously 
> compare different approaches to personalization, as the function of 
> the resulting systems is, by their nature, heavily influenced by the 
> behaviour of the users involved in trialling the systems. To-date this 
> topic has received relatively little attention. Developing comparative 
> evaluations in this space would be a huge advancement as it would 
> enable shared comparison across research, which to-date has been very 
> limited.
> Taking inspiration from communities such as Information Retrieval and 
> Machine Translation, the first EvalUMAP Workshop seeks to propose and 
> design one or more shared tasks to support the comparative evaluation 
> of approaches to User Modelling, Adaptation and Personalization. The 
> workshop will solicit presentations from key practitioners in the 
> field on innovative approaches to evaluating such systems and will 
> provide a forum to start scoping and designing tasks for the following 
> year. The resulting shared task(s) will be accompanied by appropriate 
> models, content, metadata, user behaviours, etc., and can be used to 
> comprehensively compare how different approaches and systems perform. 
> In addition, a number of metrics and observations will be outlined, 
> that participants would be expected to perform in order to facilitate 
> comparison.
>
> The planned outcome of the EvalUMAP Workshop 2016 will be a roadmap to 
> develop initial shared task(s) that will be published well in advance 
> of UMAP 2017, giving an opportunity for participants to test and tune 
> their systems and complete the task in order for comparative results 
> and associated publications to be prepared for and presented at the 
> EvalUMAP Workshop in 2017. We envision that EvalUMAP 2017 will be the 
> starting point for an annual comparative evaluation challenge at 
> future UMAP conferences.
>
> *Workshop topics are evaluation focused and include, but are not 
> limited to:*
>
> - Understanding UMAP evaluation
> - Defining tasks and scenarios for evaluation purposes
> - Identification of potential corpora for shared tasks
> - Interesting target tasks and explanations of their importance
> - Critiques or comparisons of existing evaluation metrics and methods
> - How we can combine existing evaluation metrics and methods
> - Improving on previously suggested metrics and methods
> - Reducing the cost of evaluation
> - Proposal of new evaluation metrics and methods
> - Technical challenges associated with design and implementation
> - Privacy, Ethics and security issues
> - Legal and ethical issues
> - Workshop format:
>
> This will be an interactive workshop structured to encourage group 
> discussion and active collaboration among attendees. The workshop will 
> feature a keynote talk, lightning round presentation session for 
> position papers, multiple (parallel) breakout sessions, and a final 
> discussion session to wrap up the event.
>
> *Paper Submissions*
>
> The workshop is now accepting paper submissions. Position papers (max 
> 2 pages) describing approaches or ideas / challenges on the topics of 
> the workshop are invited. Submissions should be in ACM SIGS format. 
> LaTeX and Word templates are available at 
> http://www.acm.org/sigs/publications/proceedings-templates.
>
> Papers should be submitted in pdf format through the EasyChair system 
> https://easychair.org/conferences/?conf=evalumap2016 no later than 
> midnight Pacific Daylight Time on May 7, 2016. Submissions will be 
> reviewed by members of the workshop program committee. Accepted papers 
> will be included in the UMAP 2016 EvalUMAP workshop proceedings, which 
> will be indexed with CEUR. Authors of select papers may be invited to 
> contribute to a journal publication which describes the outcomes of 
> the workshop.
>
> *Important Dates*
>
> May 7, 2016: Deadline for paper submission (midnight Pacific Daylight 
> Time)
> June 1, 2016: Notification to authors
> June 7, 2016: Camera-ready paper due
> July, 2016: Full-day Workshop during UMAP
>
> *Further Information*
>
> Further information is available on the workshop website at 
> http://evalumap.adaptcentre.ie/ or by emailing the workshop organizers 
> at evalumap@adaptcentre.ie.
>
> *Workshop Organizers*
>
> Owen Conlan, Trinity College Dublin, Ireland
> Liadh Kelly, Trinity College Dublin, Ireland
> Kevin Koidl, Trinity College Dublin, Ireland
> Séamus Lawless, Trinity College Dublin, Ireland
> Killian Levacher, Trinity College Dublin, Ireland
> Athanasios Staikopoulos, Trinity College Dublin, Ireland