Lora Aroyo is a Full Professor in Computer Science. Currently, she is a visiting scholar at the Columbia Data Science Institute at Columbia University, New York. She is also Chief of Science for a NY-based startup Tagasauris, which works on hybrid machine learning and human-assisted computing strategies to enrich multimedia (e.g. video, images, and text) with meaningful information about its content, and ultimately improve video search and discovery.

Lora is an active member of the Human Computation, User Modeling and Semantic Web communities. SheShe is an ACM distinguished speaker and the president of the User Modeling community (UM Inc). UM Inc serves as a steering committee for the ACM Conference Series “User Modeling, Adaptation and Personalization” (UMAP) and is part of both SIGCHI and SIGWEB. In her role of UMAP steering committee chair, she also participates in the ACM SIGCHI conferences board. Since 2010 she has actively worked towards shaping the concept of “User-Centric Data Science“, which ultimately led to the forming and also heading of the User-centric Data Science group at the Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands.

As an expert in user-centric data science, Lora initiated the vision of user-centric experimental lab for computer science researchers at the VU University Amsterdam, which led to the realization in 2010 of the VU INTERTAIN Lab – the first of its kind in an academic environment. She led a large number of research projects, organized conferences, workshops, and tutorials bringing together methods and tools from crowdsourcing, human computation, linked (open) data, data gathering and analysis and human-computer interaction for building hybrid human-AI systems for understanding text, images, and videos. In this context, she has led major research projects in semantic search, recommendation systems, personalized access to online multimedia collections, and through these has become a recognized leader in human computation techniques for specific domains, such as digital humanities, cultural heritage, and interactive TV. She has led a number of research projects that focus on:

  • the understanding ambiguity and teaching machines to deal with ambiguity by applying techniques from crowdsourcing and human computation, data science, data quality assessment, and especially hybrid human-AI systems for text and video understanding.
  • applying Semantic Web technologies for semantic search, recommendation systems, event-driven access to online multimedia collections, and through these has become a recognized leader in digital humanities, cultural heritage, and interactive TV

She is a four times holder of IBM Faculty Award for her work on CrowdTruth: for crowdsourcing ground truth data in the context of adapting the IBM Watson system to the medical domain and applying CrowdTruth for capturing ambiguity for the purpose of understanding misinformation.

Check also her Twitter stream at @laroyo and also her SlideShare presentations.

Some of the notable research projects:

  • ReTV: Reinventing the TV for the Digital Age: Re-purposing and re-using digital content across Smart TVs, Web and mobile applications, social media and other emerging platforms
  • Capturing Bias: Diversity-aware Analysis of Bias in News Videos: models for bias- and diversity-aware accuracy measures for reliable and explainable big data analysis of media collections
  • CrowdTruth: human-assisted computing, specifically targeting workflows for the creation of ground truth data
    • Dr. Watson: Gamification of Ground Truth Collection for Medical Texts
    • Crowd-Watson: Framework for Crowdsourcing Ground Truth Data
    • Crowd Truth: Metrics to Evaluate Crowdsourced Ground Truth Data
  • CLARIAH: Common Lab Research Infrastructure for the Arts and Humanities
  • DIVE: Event-centric Exploration of Linked Heritage
  • Accurator: Annotating Fashion with Nichesourcing
  • SealincMedia:  Socially-enriched Access to Linked Cultural Media
  • ControCurator: discover and understand controversial topics and events
  • VISTA-TV: Combining LOD and behavioral information for TV analyses
  • PrestoPrime: WAISDA? Crowdsourcing Game for Video Annotation
  • NoTube: integration of Web and TV data with the help of semantics
  • CHIP: Cultural Heritage Information Personalization


Image result for ibm faculty award 2013 logo        Image result for columbia data science logo               tagasauris-logoNetwork_Institute_Academy_Assistants-1024x206                ads.fw_

logo_crowdtruth-02 (1)

Posted in home | Leave a comment

Stitch by Stitch: Annotating Fashion at the Rijksmuseum

Fashion can be found everywhere in museums. Fashion heritage collected over centuries: costumes, accessories, paintings, prints and photographs. But while some clothes and accessories are easily found and identified, others are obscure and require a trained eye to describe. What are we looking at? What kind of sleeve is this? Which materials and techniques have been used? More specific descriptions of the images facilitate better use of digital collections and enable users to wander through them in detail.

Modemuze is an online platform and network of 11 Dutch museums, including Rijksmuseum, with a fashion and costume collection: Amsterdam Museum, Centraal Museum Utrecht, Fries Museum Leeuwarden, Gemeentemuseum Den Haag, Museum Rotterdam, Paleis Het Loo, Rijksmuseum, Tassenmuseum Hendrikje, TextielMuseum, Theatercollectie Bijzondere Collecties UvA, Tropenmuseum, Afrika Museum, Museum Volkenkunde.

Annotating the collections

Researchers from VU University Amsterdam, Delft University of Technology and the Centre for Mathematics and Informatics and the Rijksmuseum (in the context of the COMMIT SealincMedia project) have developed Accurator: an online tool to improve the process of annotation of digital collection objects, e.g. being able to find relevant objects to annotate, annotate specific parts of an object, etc. Following ‘Birdwatching in the Rijksmuseum’, this time the Accurator tool will be used to describe fashion related objects from the Modemuze and Rijksmuseum collections.

Participants in the fashion annotation event are also invited to record their findings in the Wikipedia Encyclopedia, Wikimedia Commons and in Wikidata, Wikipedia’s open database. Wikipedia volunteers, as well as staff from the Rijksmuseum and Modemuze, will be present for support throughout the day.

Participation in the event is free, but registration is required. To register, please send an email to accurator@rijksmuseum.nl with your name and your interest in fashion. (We will take your subject preferences into account when setting up the Accurator tool.) If you have any questions regarding the event, please feel free to email them to this address.

Additional information

Image | Posted on by | Leave a comment

Exploration is the New Search @ SXSW2017

Posted in event | Leave a comment

Slides from our TEDx Navesink 2015 talk

Posted in conference, crowdsourcing, event, keynote, presentation | Leave a comment

The CrowdTruth Journey @VU Faculty Colloquium

Posted in event | Leave a comment
Posted in event | Leave a comment

A twitter overview of the IUI2015


Posted in Uncategorized | Leave a comment

Crowd Truth, public release v.1

Last week, 13-Jun-2014, we published the first public release of the Crowd Truth framework. The CrowdTruth description document will give you an overall view of the framework components. The CrowdTruth Guidelines document will help you logging in and using the system. On github you can find the CrowdTruth Code and future updates.

Please, be aware that this is still work in progress, and not all things work perfectly – so don’t give up right away and send us email with feedback, so that we can repair the bugs. If you have any problems using it, please contact us Lora Aroyo and Chris Welty and we will be happy to help you through the system.


Posted in Uncategorized | Leave a comment

Crowd-Watson @NLeSC eHumanities Kick-off

Today at the NLeSc, during the “De Geest Uit De Fles” event six new projects kick off their research in the area of eHumanities. As part of this, the Crowd-Watson project team will collaborate with NleSC researchers in the coming 12 months for the development of the next version of the Dr. Watson, a medical detective nichesourcing game. Here you can find my slides on “Crowds & Niches Teaching Machines to Diagnose”:

Posted in Uncategorized | Leave a comment
Posted in Uncategorized | Leave a comment