Types of Questions, Typs of Data

As I re-read my post from Wednesday, I was thinking about my inital reaction to NVivo'sassumption that one must have predetermined  categories. After letting the thoughts from class and from the blog settle, I thought that actually it is important to admit that we do come into research with assumptions and things we are looking for; NVivo allows the researcher to add and change categories (I think?), so it may not be as constraining as I initially thought. I also wondered if it would be wise to use different software packages for different purposes and questions. Woods and Dempster's (2011) article was nicely timed to help me think through this question.

I appreciated the authors' description of Transana, and while I don't fully understand its capabilities, the ability to look at multiple transcripts at once is intruiging. However, the big takeaway for me was the assertion that different questions call for different types of analysis, and that different data may call for different analysis software in the same way that one would use different methods for different data and questions.

Choosing and committing to a particular analytical software package is a daunting task that I don't want to make until I have a more grounded understanding of the differing affordances of each package. It seems that each has some basic functionality, but they exist because they answer different kinds of questions. Therefore, as usual, it is the context that matters; one isn't necessarily better than another - they are more or less well-suited for different purposes.

And this thought reminded me that, while I have a certain epistemologial frame that shapes the way I approach research and the methods I use, I need to keep looking for the right method for asking a specific question with X kind of data. So while Woods and Dempster focused on explaining how different questions can be answered within a particular tool, their discussion led me to think more broadly about choosing the right tool for the specified questions.

This post is kind of a ramble, but it is this way because I am at a place where I am beginning to really feel my grounding as a researcher, and I am having to make deliberate choices as I move away from my advisor's work and into my own. I think I'm going through an intellectual growth spurt, and this think-aloud post is part of that process. In any case, I really appreciated Woods and Dempster's article because, in addition to providing information about Transana and software packages in general, it got me thinking on a broader level. Apologies for the stream-of-conscioussness of this post.

Predetermined Categories and Linguistic Precision

We had a really interesting discussion about the importance of choosing the words we use for codes carefully. Words connote meaning we may or may not want to associate with our data, and being precise (and consistent) with our phrasing is important both for our own data analysis and others' interpretation and evaluation of it.

And because these choices are so important, it seems odd to me that NVivo works best when you come with predetermined categories. I recognize that I come into my research with assumptions and goals, but I think I would feel strange defining categories before I know what codes are going to emerge. When I started the research I am doing now, I thought it was going to be about a six-week professional development workshop, and at the end of those six weeks, I conducted what I thought were exit interviews and tried to start writing the paper. And it didn't work.

I have been writing and revising this paper since August 2013, and sometime in November I came to the realization that I needed to be patient and let things unfold and emerge. I sat back and watched the teachers teach and listened to their stories. I took notes and marked conversations, and only now as the courses ended, was I able to identify enlightening moments and begin to craft a more robust narrative.

This is just a knee-jerk reaction. I'm looking forward to playing with NVivo and exploring its features. Maybe it is the right tool; I certainly like to Word interface. But I don't know yet. I'll have to work with the tool I'm glad we're going to get the opportunity to explore many of them.

Quotations: Relevant and Linked Analytical Objects

I found Konopasek's discussion what CAQDAS afford generally, and what atlas.ti affords specifically intriguing and refreshing. I appreciated that the author celebrated the visible thinking a CAQDAS tool can provide, as many of our readings have focused on how the tools cannot "do the work" for the researcher. Konopasek also cautions that a researcher should not "believe in magic," but stresses that a tool like atlas.ti can provide the researcher with oppportunities to see their data in new ways.

I'd like to focus on the way a researcher chooses to pull out and mark quotations as new analytical objects, and how and whether the researcher chooses to link the quotes to other quotes and data.

Jesssica recently asked me a fairly straightforward question about how I chose certain quotes to be included in my paper. I thought it was a simple question with a fairly simple answer: I have kept track of relevant quotes as they emerge. I do an immediate annotation when I hear it, and then do a deeper analysis later, linking it to other quotes. I do this all in Microsoft Word, and Iseem to have a good memory for these particularly significant events. However, I realize that I am not going back through systematically and tagging or extracting other elements that may be relevant. Using a tool like atlas.ti would aid in the systematic nature of the process, and may bring out more insights than I am seeing when I am so immersed in the data. It seems that these tools could let me step back and visualize how different pieces of data connect (or do not) and an even clearer and more impactful story may emerge.

Konopasek does caution that pulling too man quotes may distance them from each other, but the separation and systemaic linking between quotations may allow a narrative emerge that is not visible when one is marking quotations as they hear and experience them.

The author begins and concludes with the notion that thinking is inseparable from doing. This is something I wholeheartedly believe, and I think it needs to be stressed that context is everything. The context in which a particular quotation or event occurs gives it meaning, and the reasearcher's understandings, biases, and positionality directly affect both what is extracted and how it is interpereted. Context is key, and while there may be a danger in losing that contextual factor in a CAQDAS, when used well, it may actually illuminate nuanced facets of an emerging narrative.

Maximizing App Potential

This class was really practical in nature. I really appreciated the opportunity to play with Transana. I have been using InqScribe, and for much of our exploration I didn't see too much difference. However two features may have convinced me to switch: time code links and the wave. More broadly, I think we should think more about how we can use the tools we already have or have access to to enhance our research processes and products.

I like the way Transana marks the time codes and links back to that place in the video. While this is important for knowing where in a file a particular statement lies, I'm thinking I could also use these markers to bookmark places I need to come back to for further analysis. For example, when I record meetings and interviews, I often write down time codes in my notes so that I know where a particular statement was made. I could use a special transcription file in Transana to mark these for easy access once the meeting or interview is done. This would be particularly useful in preparing presentations or short reports.

I'd also like to explore the possibilities of the wave form. It seems that this has the potential to reveal information or insights that may not have been gained by simply listening to the audio because you can see if there is a started utterance or breath. This may reveal some hesitation or anticipation that the researcher may not otherwise have noticed.

I don't have an iAnything, so many of the apps we discussed did not directly affect my research, but they got me thinking and helped me circle back to something I came into this program wanting to do. When I was teaching, I was the technology expert at the school (which says more about the faculty's lack of understanding for most things electronic or digital, and less about my technological skills).  The administration thought I was pretty innovative because I moved to a paperless classroom in my second semester. But I wasn't really that innovative; the English department ran out of paper and money in November, and I couldn't buy all of my own paper and ink with my starting teacher's salary, so I made lemonade. I took the tools I had around me (MS Office, Google Docs, Etherpad, Google Search, Edmodo, Adobe Reader and PDF maker, etc.) and learned how to maximize their use and potential.

Soon, I was being asked to show other teachers how I did such-and such, and I became very interested in teaching teachers to use the tools that exist (and are free) to enhance their classroom practice. I think we can do the same thing with the free and cheap apps and programs for research. The last three presenters all mentioned really interesting tools, and suggested we take advantage of our digital tools and spaces. I am going to explore that further, and see what potential lies in existing apps and features.

Choices in Transcription

As we have been talking in class, I have been bringing my attention to the wealth of audio data I possess. To be honest, I've kind of been purposely avoiding thinking of it because it seems overwhelming. Since June, I have collected 24 recordings of teacher meetings and 13 teacher interviews, each of which are at least an hour long. I've been keeping notes about important quotes and where generally to find them. The idea of transcribing any significant chunk of this data makes me panic a little inside.

So when I saw the title of Johnson's (2010) piece, I got a little excited. Then I read the paper and found out that, while dictation software can help change things up, ti might take even longer than the traditional method. (Incidentally, I found it amusing that he described the process of transcription "dull." For some reason, I thought that people who wrote about this topic ultimately found it stimulating in some way.)

Something else I found compelling was the way Hammersley (2010) described the choices and associated consequences of transcribing different parts of an audio file. I have taken a few courses that discussed transcription, but I didn't get a good sense of what kind of choices a researcher must make in transcription. Hammersley helped me think about the difference between transcribing a long pause or a breath, and I realized that (like many things) the answer is really contextual.

All of the quotes I have transcribed have been for the purpose of illuminating a point or highlighting a change in thinking. These have all been short quotes - small statements to prove that what I said about a teacher was confirmed by them. I've read a fair amount of papers on discourse analysis, and I haven't seen the point of transcribing the pauses, ums, and restarts in speech. Now I'm thinking about that choice and the consequences that go with it. On the one hand, if I "fix" someone's speech so that it is more fluid, the reader does not see the thinking process the speaker went through. On the other hand, leaving the speech in its original form may make the speaker feel vulnerable, and may even get in the way of the reader understanding the point, as typos have a tendency to do.

The choices a researcher makes regarding what to transcribe depend on the context of the argument. Why is this quote being used? What purpose does it serve? How does it move the argument forward? The answers to these questions will vary with every paper, so a researcher must reconsider them every time.

The framing of the move away from foudationalism really helped me think through these points. What kind of research am I doing? What is my broader purpose for conducting this research? We can move beyond the old "gold standard" and illuminate important points through the voices of our participants.

A Reflection on Connected Learning

This weekend I am attending the Digital Media and Learning conference in Boston. I am here because I am presenting the research I have been engaging in around digital badges. Digital badges are web-enabled microcredentials that contain a host of data about a learner's accomplishments. In 2012, DML launched the Badges for Lifelong Learning competition, and funded 30 projects from various fields to develop badge systems that foster some kind of learning. These systems ranged from informal to more formal learning envronments, and targeted learners as young as 13 as well as adult professionals. Our lab was charged with following these projects over two years as they developed and implemented their badge systems. We are not evaluating badges; we are drawing out appropriate practices for developing learning ecosystems using digital badges. My work focuses on projects' assessment practices, and how choices around assessment impact the broader ecosystem.

The theme of this conference is "Connecting Practices," and I am listening to a lot of talks about open learning environments and fostering what Mimi Ito calls connected learning: http://connectedlearning.tv/infographic

As I'm listening to these talks, I keep coming back to this idea of identity, and our power as researchers to shape those identities with the way we represent their quotes and actions. Certainly, as I stated in my last post, people craft the identity they want their audience to see. But those identites are also shaped by the environment in which they engage, and they go through another round of shaping when we write about them.

On the surface, the work I am doing around badges is focused on projects' assessment practices, and for a while now, I have been doing my best to synthesize the practices across the projects and draw conclusion. This has been a tough task, and as we near the end of this project, I have found myself having to really push to get it done. But on the plane to Boston I did a lot of thinking about my position as a researcher and an educator. One important function we as educators and as researchers can serve is to empower voices, and I spent a fair amount of time thinking about how I have and how I might do that. I thought about my work with teachers, and I thought about my presentation this weekend, and I tried to think about what connected these projects. Initially, my personal research and the badges work seemed tightly connected, but as the work has gone on, that connection has become less and less apparent.

And sometime late last night I realized that the thread that runs through all of this work is this notion of connected learning. What I strive to teach in my professional development work and what I keep finding in my favorite badges projects is the way educators can balance the elements of connected learning. All of this work brings together a community of learners around a shared interest in an openly networked space to collaboratively make sense of complex concepts and ideas.  I also realized that my excitement around this kind of learning is not new; this is the kind of learning I worked to foster in my own classroom, and it was in these kinds of environments that I was able to shine as a learner. This is the kind of learning I have always been passionate about, even if I didn't always have this vocabulary.

The reason, it seems, that the badges work had become so laborious, is I had lost sight of my original goal in the badges project: to relay the narratives around projects' efforts to foster connected learning through the use of digital badges. We've been writing for a while that "it's not just about the badges," but I forgot to step back and look at the learning ecosystems as a whole. The ones who foster connected learning have found a way to balance the formative and summative functions of assessment, and have transformed their learning ecosystems into dynamic spaces where communities of learners can engage with one another in personally meaningful contexts to collaboratively solve problems.  And while the context of the professional development work is different, the goal is the same. I need to make sure this thread is clear in all my work. When I left the classroom, I told my students that they inspired me, and that I wanted to teach people to foster the kinds of learning evironments in which these students could shine. They help me keep things in perspective and remember that I have a responsibility to show educators a way to foster learning environments in which everyone can succeed and shine.

I have an excellent opportunity to highlight the way connected learning can give learners across ages and domains agency in and power over their own learning. I have the opportunity to make a case for fostering connected learning in broader and more formal contexts. And I have the opprotunity to craft my narratives in a way that highlights the important work the badging projects and the teachers with whom I work are doing. They have crafted identites around their work, and I can provide a forum where they can show their work. If I write carefully, I can let them tell their own stories and empower them with a voice. And the people who foster these kinds of environments allow learners to develop their own identities and have agency over their own learning. That is exciting.

Searching for a "true" identity or experience

This week we talked a fair amount about "true" identities and "real" experiences in online spaces. I'd like to use this post to explore those notions further.

We all craft our identities to some extent depending on the situation and circumstances in which we find ourselves. We reveal what we want people to see, and in every space there in conduct which is socially acceptable, and conduct that is not. This seems to be true in off- and online spaces. On Facebook, we share things we want other people to know; our photos and comments reveal a persona that may not tell the entire story of a person's "actual" profile. But what they reveal is still part of who they are. 

So when we conduct research online, I'm not sure the question of whether the participant's identity is "true" or "real" is the right question to ask. This question seems to imply that the researcher is searching for an empirical truth, and suggests that there is indeed one truth to find. At least in the work I do, and it seems in qualitative work more generally, this is not our question nor our task. We are not looking for one truth, but a story, a narrative, an account of an experience that we can explore, analyze, discuss, and share with our peers. Through this sharing process we invite discussion and further analysis.  And we give voice to stories, experiences, and people, which/who may have been silenced for some reason.

Perhaps our question should focus on the experience of a person or community, and our task should be to represent that experience in a way that tells their story and moves our thinking around that topic forward. The experience that is revealed in an online setting is a real experience. It happened. Discourse unfolded and people reacted. That is real. That is valid. And that should be brought to light. A researcher can simply give voice to the experience, or they can analyze it and compare it with other stories. But it remains that the experience that unfolded in that online space is true and real.

With the proliferation of networked access and discourse in online community spaces, it seems that people are becoming more and more comfortable engaging with others virtually. This is further reason to take people's identities and experiences as true. We may not see the whole person in an online experience, but then again, we may never see all of the facets of a person's life and personality no matter how much we interact with them on and offline.

The hard sciences have shaped much of our language around research and study, but we should be careful to use linguistic precision, as our language reveals a lot about our underlying understandings of science, research, and knowing. In more qualitative work, our research may well benefit from shifting our focus to experiences rather than truths and generalizability. Much can be learned from reading, comparing, and analyzing people's experiences.

Privacy, Lurking, and Deception/Identity Formation in Online Spaces

Garcia et al. (2009) bring up many important and interesting issues that arise when an ethnographer moves their practice to  online spaces. While the authors are specifically addressing ethnographic practice, it seems that most - if not all - of the issues they raise are applicable to research with subjects in online spaces generally. Three points stood out to me as ones I want to explore further: privacy, and lurking, and deception/identity formation.

Privacy
Depending on the site within which a researcher is conducting their study, information may seem inherently more public or private. However, this line is quite blurry, because most often people posting content to a site are writing to a specific audience, and while that audience may share this information outside of the site, the author may not have intended their content or discuorse to be analyzed. In this way, it would seem that a researcher should err on the side of privacy.

But what about sites where the subjects know there is a researcher, but become comfortable enough to interact as if the researcher were not there? I'm facing an interesting issue in my own research at the moment that makes this point particularly salient. The teachers I have been working with know that I am collecting their conversations and lessons for analysis, but they are working in a private and intimate space. They have revealed things about themselves as people and as teachers that, taken together show an array of experiences. However, I am concerned that when I write up their experiences, they may feel vulnerable, as I will have exposed their more and lelss successful moments to illustrate tthe journey each teacher took as they designed using new techniques. I have done all that the IRB asked me to do, but I'm wondering if it is enough. I am concerned that the things that they revealed in this semi-private space may seem too exposed when I write about them.

Part of my concern may stem from the way I have interacted with these teachers and their classes, which brings me to:

Lurking
While we have regular meetings via Google hangout, I am also looking in on their courses. However, while I am listed as a "teacher" in two of the three courses, I have no interaction with the students, and I can look through student work, forum conversations, gradebooks, and anything else on the site undetected. The teachers and the students have been informed that the student work and discourse will be collected, but it still feels very strange to lurk in these courses. I feel almost as creepy as the term sounds. Because of this, I've made a point to only check in periodically, and when I do, I try to engage the teachers in some kind of conversation before and after I go in, so it feels like I am having a productive discussion rather than sneaking into a classroom.

Garcia et al. Begin to address this point, but they largely leave it up to the researcher to decide how to handle the situation. When and how much lurking is acceptable? Jenkins (2009) encourages lurking in a participatory environment, but as I am not a part of these classroom communities, I am reluctant to lurk too much. This has led to a reliance on the teachers' questions and reflections on their classroom experience. Somehow I don't think I'll feel so intrusive once I am looking at the data after the courses have ended, but I'm not positive about that.

The researcher-as-lurker also brings up an interesting and important power issue. If a researcher just lurks, they know more about the participants than the participants know about them, and that puts the researcher in a powerful position of being able to analyze, characterize, and possibly manipulate the subjets. Even if they don't lurk the whole time, participants may balk at the once-hidden identity, as garcia et al. Describe in their examples.

This leads into the last point I want to explore, which is forming an online identity as a researcher.

Deception/Identity Formation
As a researcher, I have a responsibility to provide my subjects with as much information as possible to help them make an informed decision about whether or not to participate. And maybe this is where ethnography differs from other kinds of research, but it seems unethical to hide information and motives from my research subjects. Not only do I tell my research subjects about my professional experience, I also tell them bits and pieces of my personal interests so that they can relate to me a a researcher, teacher, and human being. I understood Garcia et al.'s point that we all craft our identities to a certain extent, whether we are working on or offline, but the way this point was presented made me wonder if they were implying that it is ethical to craft one's identity as a researcher. Their example of the female researcher using the username "Copperhead" made me stop and think because I can understand the motives for choosing a more aggressive and male-sounding name, but that name presents a deceptive identity that I am not sure is entirely ethical.

It seems that participants really want to know the person with whom they are working. One teacher asked me outright how old I was and how long I taught. Whe I told him these things, his reponse was that he has been teaching for 28 years and has seen a lot of fads in that time, implying that this work is one of those fad that will go away soon. At the time, I wondered if I should have asked my advisor - a 52 year old male - to be more present, but as I've thought about it, I've realized that my identity as a young innovative teacher and research have been important in shaping the kind of discourse our small community has engaged in.

These three points all intertwine, and have made me think hard about the ethics of my own research. Garcia et al., as well as others we have read, readily point out that online research is a budding practice, and the ethical implications of this kind of research are still being explored. Maybe the IRB hasn't thought about all of the implications; if that is the case, I feel it is my responsibility as a researcher to consider them.