This is the edited script of a presentation given by Karine Larose and Simon Barron at UXLibsIII conference in Glasgow, UK on 2017-06-06. It has been edited to incorporate the accompanying presentation into the text and to insert additional detail and quotes. Thanks to the organisers for selecting our paper and allowing us to present.
Addendum (2018-03-06): This text was written up and published as a book chapter in the User Experience in Libraries Yearbook 2017 edited by Andy Priestner. The full text of the book chapter is available at http://eprints.rclis.org/32461/. It is © Karine Larose and Simon Barron and is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
1.0: introduction
During summer 2016, Imperial College London’s Library Information Systems team ran user experience research into the information-seeking behaviour of undergraduate and postgraduate students focusing on the use of the library catalogue and discovery interface. We gathered some really interesting findings which are helping to inform the design of our library catalogue. Our results are available in reports at http://hdl.handle.net/10044/1/44345 and http://hdl.handle.net/10044/1/44390 (licensed as CC-BY 4.0) but today we want to talk about what we did wrong, the limitations of our methodology, and the impacts of that on our approach to inclusion and diversity in our UX work and our view of wider UX research in libraries.
Our research methodology used semi-structured interviews and comparative think-aloud exercises underpinned by elements of the grounded theory approach to data heavily influenced by Kathy Charmaz’s (2014) methodology outlined in her book, Constructing Grounded Theory. Grounded theory is a methodology for data collection and analysis: “Stated simply, grounded theory methods consist of systematic, yet flexible guidelines for collecting and analyzing qualitative data to construct theories from the data themselves. Thus researchers construct a theory ‘grounded’ in their data.” (Charmaz, 2014: 1) Grounded theory is simply a set of systematic but flexible guidelines for collecting and analysing qualitative data from UX research. The method keeps the researcher close to the data by stripping away their assumptions, their preconceptions, and other baggage that researchers bring to the data. In theory, it therefore produces findings which are truly grounded in the data rather than in the preconceptions of the researchers.
Our published reports are therefore only half the story: the ‘objective’ acceptable face of the story. The other half is the personal half. The grounded theory methodology for data analysis helped us acknowledge some of our unconscious biases and assumptions and during our analysis, we discovered problems with our approach to inclusion and diversity. More problems than we had space to adequately discuss in an evaluation section. We had failed to get a representative sample of our diverse student population, we hadn’t tried to attract or make accommodations for users with disabilities, and, at times, we hadn’t treated the research with the ethical responsibility that it deserved.
So our personal evaluation extended beyond the published evaluation. And we couldn’t evaluate our own UX work without evaluating the larger context within which it took place: a specific organisation working to a specific paradigm of library UX work with a specific library UX community taking place within a society of white supremacist capitalist patriarchy. We also needed to think intersectionally about various reasons for prejudice against users: although focused on disability, we couldn’t think about disability without also thinking about how it connects to gender prejudice, class prejudice, racial prejudice.
Critically reflecting on our experiences in context with our personal experiences and the environment in which we operate has been useful when evaluating our UX research. Fook (2007) describes critical reflection as the “ongoing scrutiny of practice based on identifying the assumptions underlying it.” We are hoping that this will transform our practice as UX researchers and create positive change. In our view, the desire to create positive change following critical reflection requires a willingness to make a conscious efforts, a readiness to challenge, and the courage to stand your ground when encountering resistance.
As well as using our critical reflection and evaluation of our own research to talk about improving our UX practice, this paper takes something of an autoethnographic approach (Ellis et al., 2010) describing the experiences of Karine Larose, a woman of colour with a disability, conducting UX research. The beginning of our process of change started with her traumatic experiences working in the library profession and in her UX research practice, recovering from her traumatic experiences, and taking conscious actions to lead on change.
2.0: background
Imperial College London is usually described as an international university. We have students studying at Imperial who come from all over the world. In general, the number of international students exceeds the number of British students. This table shows the diversity of undergraduates and postgraduates at Imperial College London: we’ve deliberately removed the numbers of students of each nationality because numbers shouldn’t matter when it comes to diversity but each one has more than 250 students for 2017-2018.
| Country | Language/s |
| UK | English |
| China | Chinese |
| France | French |
| Malaysia | Malay, Malaysian, English |
| Italy | Italian |
| Singapore | Malay, Mandarin, Tamil, English |
| Germany | German |
| Spain | Spanish |
| Greece | Greek |
| India | Hindi, English |
There are also lots of students with disabilities at Imperial: both visible and invisible disabilities. When conducting user experience research and designing both spaces and systems, we think that using concepts from the social model of disability can have a positive impact. The social model of disability sees ‘disability’ as socially constructed (Scope, 2017). Many of the problems that people experience that we associate with ‘disability’ are actually created by the social environment around them. “In our view, it is society which disables physically impaired people. Disability is something imposed on top of our impairments by the way we are unnecessarily isolated and excluded from full participation in society. Disabled people are therefore an oppressed group in society.” (Union of the Physically Impaired Against Segregation, 1975)
Andrews (2016: 111) is an example of a user with autism experiencing a “sensory assault” in one library: “I can smell the café, which is serving food and coffee. I can smell the toilets, which are disgusting. I can hear a thick wall of noise that buzzes around my head and makes me feel woozy. The lights are harsh.” By contrast, in another library which has eliminated all those things, they function perfectly well. In one environment, they were ‘disabled’; in another, they weren’t.
By focusing on the environment rather than the user, the social model brings new thinking and opens the path for positive transformations. We want to design library services which allow all users to fully use them and feel fully included. The social model shifts the focus from ‘integration’ – making specific adjustments for individual students – to ‘inclusion’ – anticipating and accommodating a range of possible needs (Belger & Chelin, 2013: 8). Using the background of the social model of disability, you can make your user experience research more inclusive and make the outcomes more favourable to a wider variety of library users. This benefits not only the disabled but also the non-disabled.
3.0: diversity in UX research
“UX is for everyone, not just those who are deemed to be the majority group. Everyone is entitled to a good user experience, and no user is ‘lesser’ than another.” (Andrews, 2016: 108). UX work, particularly quantitative UX work, often runs the risk of slipping into majority rule. For library systems suppliers, user experience research is often based on gathering the needs of users and averaging them out resulting in lowest-common-denominator systems that appeal to the greatest number of users. In library UX, we do this by modelling services and systems based on the reported experience of the majority of users. We “continue to design services for one majority group (e.g. ‘18-25-year-old students’) and treat other groups such as disabled users, part-time students, older users, non-native English speakers and so on as add-ons – the ‘non-traditional students’ or the ‘socially excluded’.” (Andrews, 2016: 114) If 20 undergraduates want a single search box and 10 researchers want an advanced search box, then we strip out the advanced search to meet the needs of the majority. We end up with library systems that meet the needs of a large number of white neurotypical non-disabled undergraduates but fail to address the needs of other categories of user.
There are lots of ways to take consideration of inclusion and diversity in your UX work. These are just a few.
3.1: recruitment
First, recruitment and selection. Do you feel uncomfortable talking to a disabled person or other people from marginalised groups? Have you avoided talking to disabled people for fear of saying the wrong thing?
A library doing UX work needs a framework for how to recruit UX participants with consideration given as to how to attract participants from minority groups. Attracting diverse participants is the first step towards hearing their voices. It’s particularly important to consider the role of unconscious bias in selecting participants: generally we are unconsciously biased towards people who look like us. White people are more inclined to work, talk, and interact with people similar to them. White men are more likely to cite other white men or even themselves (King et al., 2016). At Imperial, the library staff are mostly white, neurotypical, and non-disabled and this generally leads to recruiting white, neurotypical, non-disabled users to participate in our UX research. For example, with the grabbing-students-who-happen-to-be-around method of recruitment, there is a high chance your unconscious bias is influencing who you pick. This way of recruiting users can be quick and easy but it creates an exclusive structure where your users from marginalised groups are at a disadvantage. Even if it doesn’t come from bad intent, we need to acknowledge the existence of our unconscious biases and how they affect what we do as UX researchers. We need to actively work on ways to eliminate them from our UX practice.
Sending emails or putting out social media messages is a better way to reach a diverse audience. Think about if your message is visible enough to underrepresented groups. Is the language used in the message inclusive? For the last UX research into library space at Imperial, Karine proposed we change the phrase “walk around the library” to something more inclusive like “explore around” or “tour around” to be more inclusive of wheelchair users. In your messages, be clear, concise, and outline exactly what is going to happen in the research process. Mention how the research will be accessible for users with disabilities including users with visual impairments, wheelchair users, and users with invisible disabilities. Our HR departments routinely do this in their recruitment processes so we can do it in our UX research processes.
Our recent UX project failed at recruitment primarily because we sent emails through only one channel: our Liaison Librarians team. We should have recruited through a range of channels rather than just the library and attempted to connect to the wider university community. Some good places to recruit through include the various departments of the university, the Students’ Union, the university’s Diversity & Inclusion Centre, and student newspapers.
3.2: research design
Second, research design. When we interview participants, we need to be mindful of factors like language barriers, accents, and other speech patterns that can inhibit understanding of one another.
As mentioned, most students at Imperial are international students and English is not their first language. Sometimes users whose first language is not English living for the first time in an English-speaking country rely on verbal and nonverbal cues to understand. Just because you understand your questions, it doesn’t mean your users do. The labour of ‘understanding’ should be on the researcher rather than the research participant. If the user has to strain themselves to understand you or the questions, then you’re doing wrong and your research is inaccessible.
‘Using silence’ is a popular technique in UX interviewing as well as coaching and job interviews. Lots of methods advocate using silence as a tool to allow users to reflect on questions to make informed replies. This says that you can use the awkwardness of silence to draw people out by making them respond to end the silence. However, silence can also be a sign of badly planned and non-inclusive research design.
Poorly-framed questions can also rely on jargon such as “Library Search” or “E-shelf”. These can be a barrier to understanding and will affect how users reply to your questions. Disabled users with learning difficulties with be particularly disadvantaged. Users with dyslexia, for example, might have problems with short term/working memory which makes it difficult for them to follow or remember instructions for an insights boards exercise with multiple axises representing different concepts and multiple colours of Post-it notes representing different meanings.
Some simple efforts like testing your questions and research design on diverse staff, printing your questions out in accessible fonts for participants, and asking about access requirements can make a difference. Based on the social model of disability, making changes to cater for the needs of your disabled users will also improve your UX research which benefits ALL users.
3.3: definitive actions
As well as these general tips, we want to share three actions that we are doing to improve our UX research and that you could find useful.
First, challenge non-inclusive UX research. Unchallenged processes create an exclusive environment. If you feel that is something is not right, you can say “no”. Saying no is scary especially if you have been undermined or called names in meetings before. It can be particularly scary if the UX meetings are mainly white men talking at you and undermining you.
Karine said “no” on our previous UX project when we were rushing data analysis and summarising complex results to create a report for senior management. She said no because she thought it was unethical. She wouldn’t put her name on rushed work. When she said “no” to Simon, he made the conscious effort of finding out why she said no and he took a side. It’s not easy but it helps to challenge a non-inclusive environment.
Second, when it comes to learning in UX, be your own leader. The UXLibs conference is a space to get inspired, to learn more about UX methods, to hear the experiences of UX practitioners, and share our practice with the UX community. What we get from UXLibs is valuable but also limited. We can’t rely on it to build our entire knowledge of user experience research.
It’s important for us to reinforce my knowledge and understanding of UX research independently, even within an organisation. Do not let someone else’s interpretations or understanding define your approach to UX research. Questioning and challenging other people’s interpretations helps with innovation and creates better UX. UX leaders need to encourage active learning in their teams and be open when being challenged.
When we did data analysis using elements of grounded theory, we had three coders and it took a lot of time but we each took the responsibility to read Charmaz’s (2014) work and discuss it together. We each challenged each other’s understanding of the data to stay as close as possible to “the voice of the users” (Charmaz, 2014).
Third, a UX inclusivity framework. This spring, Karine completed the Calibre disability leadership programme that gave me the confidence to explore her own disability and provided me with theoretical insights. It also gave her a practical guide to deal with disabling barriers in the workplace. As part of her personal project from this programme, she will be developing an inclusion framework which will ensure our UX research recognises the various needs of all our users. This will hopefully bring a positive change to the services we provide in the library.
4.0: diversity in UX community
Of course, diversity starts with the team doing the research. The staff working on UX need to have a good understanding and appreciation of diversity considerations or all of the practices we’ve outlined will just be lip service. It’s not enough to employ diverse staff and ask for their input in meetings if their input is just going to undermined. Personally we used to stay silent when hearing bad ideas in meeting hoping the bad ideas would crash and burn to the ground before getting applied but some bad ideas are resistant.
Managers and other workers need to keep their insecurities out of meetings, need to not undermine workers from marginalised groups, and give team members – especially marginalised team members – room to talk and contribute. We wish we had got to spend more time on focussing and reflecting of our UX practice instead of dealing with white men’s insecurities.
Working with people from different backgrounds needs to be encouraged as more than just affirmative action in LIS. We also need to acknowledge people’s differences and different backgrounds so that, for example, black workers in LIS are not forced to perform whiteness to fit in (Hathcock, 2015). This creates the space for different working styles, hearing and learning from a wider range of views. Ultimately diverse views from staff make for better decisions and better UX research.
Our UX research is also affected by the lack of diversity in the UX community and the wider LIS sector in general. The library systems domain in particular is already a very challenging place for marginalised groups and by ignoring those challenges for marginalised workers, we magnify those challenges and make marginalised groups invisible.
We love doing UX research but hate the inclusion challenges and the diversity gap. There simply aren’t enough visible people of diverse backgrounds in library UX. Karine has said “As a woman of colour with a disability, I know how hard it is to battle the diversity statistics all the way up to professional growth and positive change in an organisation.” White people are good at inventing concepts to justify their racism and finding excuses not to be inclusive. We hear leaders and professional bodies like Cilip (2015) saying we don’t have enough diversity or ‘we have a diversity problem’ and manufacturing data – mostly quantitative – to support them and illustrate the lack of diversity. But this is white people trying to justify the lack of diversity instead of taking concrete actions to include marginalised people in UX practice. When white people talk about the lack of diversity, it makes marginalised workers feel invisible. Black workers, minority workers, and disabled workers are only invisible if you make them invisible.
5.0: references
Andrews, P., 2016. ‘User Experience Beyond Ramps: The Invisible Problem and the Special Case’ in Priestner, A., and Borg, M., eds. User experience in libraries: applying ethnography and human-centred design. Abingdon: Routledge, pp. 108-120.
Belger, J. & Chelin, J., 2013. ‘The inclusive library: an investigation into provision for students with dyslexia within a sample group of academic libraries in England and Wales’, Library and information research, 37(115), pp. 7-32.
Bonnici, L. J., Maata, S. L., & Wells, M. K., 2009. ‘US national accessibility survey: librarians serving patrons with disabilities’, New library world, 110(11/12), pp. 512-528.
Charmaz, K., 2014. Constructing grounded theory. 2nd edition. London: Sage.
Cilip, 2015. ‘Landmark UK information workforce survey reveals ongoing gender pay gap’ Cilip: the library and information association. 24 November 2015 <https://www.cilip.org.uk/news/landmark-uk-information-workforce-survey-reveals-ongoing-gender-pay-gap>
Dermody, K. & Majekodunmi, N., 2011. ‘Online databases and the research experience for university students with print disabilities’, Library hi tech, 29(1), pp. 149-160.
Ellis, C., Adams, T.E., and Bochner, A.P., 2010. ‘Autoethnography: an overview’, Forum qualitative sozialforschung, 12(1), pp. 345-357.
Fook, J., 2007. ‘Reflective Practice and Critical Reflection’ in Lishman, J., ed. Handbook for practice learning in social work and social care, second edition: knowledge and theory. Basingstoke, Palgrave, pp. 363-375.
Hathcock, A., 2015. ‘White Librarianship in Blackface: Diversity Initiatives in LIS’, In the library with the lead pipe, 7 October 2015 <http://www.inthelibrarywiththeleadpipe.org/2015/lis-diversity/>
King, M., et al., 2016. ‘Men set their own cites high: Gender and self-citation across fields and over time’, arXiv, 30 June 2016 <https://arxiv.org/abs/1607.00376>
Larose, K., et al., 2017. ‘Library Search UX report summer 2016’. Published 13 February 2017 <http://hdl.handle.net/10044/1/44345>
Scope, 2017. ‘What is the social model of disability?’, Scope <https://www.scope.org.uk/about-us/our-brand/social-model-of-disability>
Stevenson, L., & Larose, K., 2017. ‘Library Search UX survey results 2016’. Published 13 February 2017 <http://hdl.handle.net/10044/1/44390>
Union of the Physically Impaired Against Segregation, 1975. ‘Fundamental Principles of Disability’, discussion held on 22 November 1975 <http://disability-studies.leeds.ac.uk/files/library/UPIAS-fundamental-principles.pdf>