How to make sense of the Leximancer analysis: A quick guide to interpreting concept maps Academic Enhancement Unit, 5th Floor, Kingsway House, Hatton Garden, Liverpool, L3 2AJ This quick guide has been developed to support the research processes of those who are new to the semantic analysis software Leximancer and would like to make a prompt start with interpreting the data. the frequency with which each word is used and then calculates the distance between each of the terms (co-occurrence). The algorithms used are statistical, but they employ non linear dynamics and machine learning. The guidance is mainly focused on making sense of the concept maps and gives some tips on what to look for when exploring the data. The results of computations are displayed as a concept map that can be explored on individual concept levels and also by looking at the family of associations between different concepts (themes). Although a couple of ‘how to do’ bits of advice are also included, generally we assume that the researcher would know how to reach the ‘output’ (actual concept map) stage. This guide is based on the latest Leximancer 4 version, so the interface could be slightly different from the earlier versions of the software, but the general principles of the analysis and core functions of the software remain the same. What is Leximancer? This is a semantic analysis software that has been developed at the University of Queensland, Brisbane in 2001: (https://www.leximancer.com/). Why/when would you want to use the software? Leximancer is a useful instrument for researchers/analysts who need to explore a large text-based data set where manual analysis and coding would be too time consuming, e.g. qualitative survey data, multiple interview/focus group transcripts, lengthy reports or web-based textual information. How does it work? The process is called unsupervised semantic mapping of natural language. The method can also be thought of as a form of text mining. Leximancer employs two stages of information extraction: semantic and relational, using a different algorithm for each stage. It computes What are the benefits of using the software? Leximancer provides a fairly unbiased method of reviewing complex textual data sets and a clear process of justifying decisions about text selection. It makes the researcher aware of the ‘global’ context and helps to discover ‘hidden’ structures in text that fall outside of his/her preconceived framework. The automated analysis demonstrates a stability of measurement over time and what is most important - it allows a more rapid and frequent exploration of text with reduced cost. Tailored (researcher driven) analysis also could be done if the researcher would like to explore specific topics/concepts. One of the attractive features of the software is its ability to identify sentiments by showing probability of a concept being mentioned in a favourable or unfavourable context. Leximancer could be used on its own or as a complementary tool for other methods of analysis. Anything to be aware of/any limitations? There are some limitations to ‘purely automated’ analysis. Some concepts emerge strongly where they are represented by a narrow vocabulary. Others will be identified from a broader pool of 1 terms and have a greater likelihood of being diluted as concepts in the map. This can be mitigated by the manipulations with a thesaurus (e.g. adding concepts that haven’t make a relevance threshold to the concept map, creating combined/compound concepts etc) and should be acknowledged as a researcher-driven analysis. Leximancer findings might benefit from being combined with outcomes from other type of analysis such as traditional thematic analysis or content analysis for gaining a more comprehensive picture. Some more insights into the software and how it works can be found on: http://www.textinsight.net/sites/default/files/ files/What%20is%20Leximancer.ppt The most exciting part is actually using the tool! First of all... First of all you need to run a preliminary analysis of your ‘unspoiled’ data set, with no editing or configuration, to get a ‘feel’ of the data. If you are using the software for the first time, for a step by step guidance of how to make a start look at the PowerPoint presentation created by Julia Cretchley and Mike Neal: http://www.textinsight.net/sites/default/files/ files/First%20Leximancer%20Analysis.ppt How to make sense of your first concept map Concept level exploration 1. Take some time to visually explore the concept map generated. You might want to set the Theme Size scrolling Bar on 0% (see Fig. 1), so that you can focus on the concepts first. Fig 1. 2. When you have located the most relevant concept (largest circle), you will instantly know what is of the most importance to either your respondents/research participants or to the text creators. Pay attention to the concepts positioned closely to the most relevant concept those with direct links/connections would indicate that these two words are often used together and worthy of detailed exploration. Things to note: Watch out for language your research participants use. Students often use different terms to describe the same concept - it might give you an indication of a specific connotation or a context in which they choose to use a particular word. For example while describing academic staff, students use different words such as staff, lecturers, tutors, teachers etc. What is interesting is that these concepts/themes are not always located close to each other and associated with a different contextual and often emotional background. 2 3. Explore other concepts (those that constitute ‘nodes’ in the concept map). Their relevance is reflected by size (more relevant ones have bigger size). be shown via graphical links and also as a table indicating the likelihood of other concepts being mentioned together with the concept in question (Fig 3). Note where they are positioned on the map and which concepts they are surrounded by. This could give you an idea of other significant themes and how they are connected. 4. Look for concepts that are less important/ these could be excluded from your analysis, (these, for example, these could be frequently mentioned words such as ‘certain’, ‘have’ etc). Some concepts could be merged (e.g. placement and placements) or made compound (e.g. work + experience). You could make these changes later to make your concept map more readable. 5. Explore the Concept Ranking Table located to the right from the concept map (Fig 2). The relevance of the concepts will be indicated in the table numerically (number of instances/actual quotes and relevance %). Fig 3. 7. When you have familiarised yourself with the layout of the map and have a broad understanding of the concepts that are coming out from the data and how they are connected, it’s time to explore the direct quotes/instances that formed these concepts. Go to the Concept Ranking Table (located to the right from concept map) and under the Count heading click on a number that corresponds to the concept you would like to explore (Fig 4). You will have instant access to all the quotes that contributed to the creation of the concept. Fig 2. 6. By clicking on a concept on the map you can see how this concept is related to others. It will The quotes will give you an indication of the meaning(s) behind the concept and essentially how ‘semantically clean’ is the concept. For example the word ‘feedback’ would most likely have a singular meaning (Fig 5), while ‘work’ might have multiple meanings, since students, for example, could talk about their course work, part-time work, work related learning, work as verb – e.g. ‘work it out’ etc. 3 In the case of multiple meanings you might want to explore only specific links/connections with other concepts that are of interest for your research, or make some adjustments on the next advanced stage of the data processing. Things to note: Not all possible quotes where the concept (word) was mentioned in the text will be included in the list. Only those that passed the relevance threshold will be listed. (Fig 5) 8. If you are interested in quotes/instances where two specific concepts were mentioned together, click on a particular concept (under WordLike) in the Concept Ranking table, and when a list of related concepts appeared with the icon , click on the icon attached to the concept that you would like to explore together with the first concept. (Fig 6) Fig 4. above and Fig 5. below. Fig 6. Exploration of themes Themes are concept clusters that represent the most semantically connected groups of concepts. Theme name is the most prominent concept in the cluster. 1. Set your Theme size scroll bar to 100% - to see the concept(s) with the highest level of connectivity – these will be the most important themes that are coming out from your text (Fig 6). There might be one theme, two or, sometimes, more – they are all worth paying close attention to. You could explore relevance of the themes and as in the case of concepts, have access to all quotes that are illustrative of the theme. 4 Things to note: If you‘ve got more than one theme at 100% resolution, look for the concepts that are positioned in the intersected area(s) – they might give you some insights of what topics ‘belong’ to both themes. Fig.8 Next step… Fig. 7. 2. When reducing percentage on the scroll bar, more themes (concept clusters) will appear on the map (Fig.8). Look carefully at the dynamics of transformation - e.g. how smaller themes merge into bigger ones, what themes disappear and what remain. 3. Remember or better save the themes generated by the automatic analysis. It is possible that when you made some custom configuration changes, the thematic picture will also change. The next step is making necessary adjustments for a more focused/customized analysis, for example, removing concepts that are not central for your research, merging or adding the concepts, activating sentiment analysis, adding tagging etc. When all the changes you think might be useful for your research are done, the analysis should be run once more, following the same stages that were indicated/described above. Sentiment analysis Activating the sentiment lens can be a bit tricky, so below is a quick step by step guide how to do it. 1. To enable sentiment analysis, in the Project Control Panel select ‘Generate Thesaurus’ and then activate ‘Show Settings’. 2. Click on Concept Seeds’ and then on ‘Edit’ (Fig 9). 5 The latter case will be evidence of a particularly strong disposition of your research participants. By clicking on favourable/unfavourable you will have access to all quotes that were selected as indicative of the particular sentiment for this concept. Things to note: Don’t be surprised if, for example, the percentage of unfavourable likelihood is higher, but the number of quotes selected by the software will be the same than in case of favourable quotes. Fig. 9 3. Choose User Defined Concepts from the top menu and then click on Sentiment Lens button that is activated/highlighted. Favourable (favterms) and Unfavourable (unfavterms) terms will appear under the Concept list (Fig 10). 4. Save the settings by pressing ‘ok’ and then run your analysis from scratch so that sentiment analysis is incorporated. This is not a direct/proportional relationship, as it is determined by various statistical procedures and indicative of more complex relationships between the concept and strength of the sentiments expressed. Things to note: Pay attention to the location of favourably and unfavourably rated concepts on the map. Sometimes the same sentiment concepts cluster together, this would be an indication of strong shared concerns/issues. It is also useful to explore themes in relation to dominant sentiment background of the concept that form the theme. (Fig 11) Fig.10 Why use the sentiment analysis? Sentiment analysis gives you an instant indication of probability of a concept being mentioned in a favourable and unfavourable context. Just click on a concept in the Concept Ranking Table and see the results. The difference could be relatively small (as in Fig 11 – just 1% or under) or quite noticeable. 6 Final remarks References You could use some advanced functions useful for the analysis: for example tagging files and groups of files to allow for comparison, creating Dashboard Reports, explore word cloud, look at your concept map from the ‘social network’ perspective etc. 1. Smith, A. & Humpreys. M., 2006, ‘Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping’, Behavioural Research Methods, (38), pp. 262–79. Instructions how to do it can be found in the ‘Help’ section of the software. Some advance features are also explained in: 2. Dodgson et al., 2008 ‘Content analysis of submissions by Leximancer’. Available at http:// www.innovation.gov.au/Innovation/Policy/ Documents/LeximancerSubmissionAnalysis.pdf (accessed 15 April 2013) https://www.leximancer.com/dl/training/201112-V4194059105-AB-45-J/LeximancerIn-depth.ppt The main and most time consuming work will be about delving into the actual quotes and extracting meanings of the concepts and concept connections, making sense of themes and general landscape of your text. When you have had a chance to compare the outcomes of a ‘fully automated’ analysis and a researcher-driven, customized one you might decide that automated analysis is more insightful. It is easier to start everything from scratch (as a new project) rather than trying to ‘undo’ the changes that were made. Comparative analysis of concept maps Leximancer could be a helpful tool when you need to compare two data sets or to look at the data longitudinally. Look for new, emerging concepts and concepts that have disappeared, explore how the relevance of the concepts and themes have changed, what happened to direction and strength of the sentiments for the key concept, how positioning of the main concepts and their neighbours changed. Comparative analysis could be especially insightful! Please send your questions or comments related to this guide to Dr Elena Zaitseva ([email protected]) 7