Hi folks,
This is Dave Mazella, posting a follow-up to Anna and Eleanor’s previous discussion of teaching with ECCO. As we talked about pedagogical strategies for including ECCO in eighteenth-century courses, the question arose of how one might assess these kinds of activities and their impact on student learning.
Julie Grob, a UH special collections librarian and a collaborator of mine, has generously agreed to share this IL pre-course assessment that she designed for a research-intensive course we developed together. This kind of assessment, taken at the beginning and end of the semester, can help you assess the impact of a semester’s work in primary sources. These questions were administered through surveymonkey.
The background to the course can be found in this co-written article we published in portal, a scholarly library journal available on JSTOR and Project MUSE. Julie developed these questions as we both worked through the ACRL Research Competency Outlines, which were very helpful for designing both assignments and assessments.
***
1.
- Have you previously taken ENGL 3301, Introduction to Literary Studies? [this is my Intro the Major course, which includes some work in Spec Collections]
- Have you ever visited Special Collections, either with a class or on your own? If the former, for which class?
- materials from the 18th century only
- the first sources you should look at when doing your research
- sources that contain contemporary accounts of an event, written by someone who experienced or witnessed that event
- any sources held by a library, regardless of format
3. From the answers below, which is the best definition of secondary sources?
- any materials held by a library that are not rare
- sources that are not relevant to your particular research
- sources that interpret an event, written by someone at least one step removed from that event
- any materials that were published after the 18th century
4. What kinds of materials are found in the UH Libraries’ Special Collections? (Please check any that apply).
- old books
- new books
- journals/magazines
- newspapers
- maps
- letters
- documents
- photographs
5. How would you find out if a book about Benjamin Franklin is located in Special Collections?
- Come to Special Collections and look at the paper card catalog
- Come to Special Collections and wander through the book stacks
- Search for books about Benjamin Franklin in the library catalog, then “limit” your search to Special Collections
- Search for Benjamin Franklin under “archival finding aids” on the Special Collections website
6. Which of the following are common features of an 18th century book? (Select four).
- printed on vellum (animal skin)
- printed on paper
- bound in leather
- bound in colorful bookcloth
- illustrated with engravings
- illustrated with photographs
- words have a “long s”words have a “double y”
7. What kind of source would be most important for a scholar to consult if he or she wants to do original research (that is, research that creates new knowledge in their field)?
- an electronic source
- a primary source
- a secondary source
8. Which of the following databases would be most useful for finding articles about literature? (Select three).
- ERIC
- JSTOR
- MLA
- Philosopher’s Index
- Project Muse
- PubMed
9. If you search one of the Library’s electronic databases using a keyword and get back 500 hits, how might you most effectively change your search to get back a more manageable number of results?
- use a totally different keyword
- add a second keyword
- do a keyword search using Google instead
10. Where are you most likely to find accurate information about a famous person from the 18th century?
- Wikipedia (web site)
- MLA (database)
- Dictionary of National Biography (database)
We used this as part of our documentation of student learning for the SACS QEP, which helped fund the acquisition of some special collections material for the course.
DM
Early Modern Online Bibliography 
August 21, 2013 at 2:35 am |
Thanks so much, Dave–and Julie, too–for these questions. While we’ve used somewhat similar types of questions in our program assessment for information literacy (using the ACRL competencies as a basis), I very much like the pre- and post-measurement aspect of what you are doing here. Similarly, designing and administering brief surveys for individual courses could enhance the learning experience on several levels. Not only does it make students more aware from the outset about research and its relationship to the course, but it should also enable the instructor to approach research instruction with a far better understanding of what his or her students need.
LikeLike
August 21, 2013 at 6:12 am |
An interesting set of questions that present some basic research questions. I had not thought of using such a quiz in our intro. to literary studies course, but that is a great idea.
LikeLike
August 21, 2013 at 6:22 am |
Eleanor, I found this web site for ACRL standards: http://www.ala.org/acrl/standards/informationliteracycompetency#stan. Is this what you meant by ACRL competencies?
LikeLike
August 21, 2013 at 9:49 am |
Yes, Anna–those are the standards to which I refer. Like Dave, I’ve worked closely with one of our librarians.
LikeLike
August 21, 2013 at 9:48 am |
Hi Anna, just to clarify. This pre- and post-test was for a research intensive undergrad seminar we developed with QEP resources. I use a different assessment for my Intro to Lit Studies IL segment, which is very extensive. My problem is that students in the Intro course would only rarely turn up in the advanced seminar, so I have no control over the degree of students’ exposure to IL or spec colls work. If you read the article, you’ll see that we spent a lot of time conceptualizing what an “advanced” IL would look like in our discipline. The difficulty predicting who would be in our classroom and what they would know is one of the reasons why we embraced these kinds of assessments.
The link I provided above to the Literature in English section of the ACRL was key, because they were specific to literary study, but because we got some ACRL support, I believe, we also worked with the more general ones you found. We were fortunate to encounter early on Kathleen Kluegel, a research librarian at UI Champaign/Urbana, who had worked on these, and she gave us some good advice.
As Eleanor points out, these kinds of on the spot assessments are useful not just because of the information they provide of students’ prior knowledge and understanding of the topic, but the way they can refocus their attention towards your instructional goal. The undergrad English major who can define what primary sources are, or give you different examples of them unassisted, are pretty rare around here, but those completing the course can usually do at least that much.
LikeLike
August 21, 2013 at 10:01 am |
We also work with both the general and the Literature in English ACRL competencies. Our students each take 3 advanced-level research seminars on a variety of topics–not all rooted in literature (we have two tracks–we are developing assessments for each track, but I am referringher to program assessment.) The assessment program we have in place now is fairly new because we revamped the major a few years ago, and the first graduating class was 2011. We use the general competencies with an eye to translatable skills.
We have a lengthy disciplinary literacy instrument as one of our measures, and the information literacy questions to which I have been referring are incorporated into this instrument. We also have a portfolio and an exit survey, and information literacy is assessed using these two instruments, too.
LikeLike
August 21, 2013 at 10:35 am |
Eleanor, this thread is threatening to go off-topic, but please consider contributing to an assessment discussion at the Long 18th. I know a number of people in 18c studies who have done such work, and it would be interesting to see how different departments are producing and assessing their majors nowadays.
LikeLike
August 21, 2013 at 10:37 am |
Will do, Dave!
LikeLike
August 21, 2013 at 6:38 pm |
This is probably off-topic, too, but since Eleanor raised the ACRL Competency Standards, I thought you might be interested to know that they are being revised. The new version is intended to be less mechanical and inventory-based, on more focused on a variety of literacies, the creation of knowledge, and collaboration. http://www.acrl.ala.org/acrlinsider/archives/7329
LikeLike
August 21, 2013 at 7:20 pm |
Thanks for this update, Julie, and for generously sharing this assessment tool.
As a special collections librarian, which research skills do you wish instructors reinforced in their courses?
LikeLike
August 21, 2013 at 10:45 pm |
Thank you, Julie. I had not yet heard about the proposed changes (the librarian with whom I work was on maternity leave this past spring), but the plans for revision make much sense. The key thrust of the revision appears to move from the static list of skills in place since 1999 to a far more dynamic view of literacies “that take us beyond textual information, that emphasize student participation in creating new content, that encourage students to develop metacognitive abilities and different parts of the brain, and that create more and deeper opportunities for collaboration among librarians, faculty, instructional designers, technologists, and students themselves” (“Rethinking ACRL’s Information Literacy Standards”). That shift seems closely tied to the active learning and engagement these primary databases can engender.
LikeLike
August 22, 2013 at 8:15 am |
In these assessment reviews, do you address Common Core standards?
LikeLike
August 22, 2013 at 10:20 am |
@Anna, not quite sure which core standards you’re referring to: Texas does have a state-wide core, but this was designed as an advanced (junior or senior), capstone-style course in the literary studies concentration of our major. Texas does not stipulate any assessment of majors using those core categories.
What we really used were the rubrics and guidelines generated by our campus-wide SACS QEP plan, which was oriented around undergrad research. The QEP, which was and is part of our assessment process, dictates that any curricular improvements sponsored by the plan be assessed. That was the really motivating factor for us, though the article gives the full narrative of course’s development.
LikeLike
August 22, 2013 at 10:42 am |
Anna, are you referring to the Common Core standards for middle and secondary schools and MLA initiatives to dialogue?
LikeLike
August 22, 2013 at 11:43 am |
Yes. The new emphasis on non-fiction reading would seem congruent both with information literacy (assessed by Julie and Dave’s survey) and with the kind of literacy students acquire by using these databases.
Using ECCO requires higher order literacy skills than, say, reading popular science magazines, a genre that might at first seem an appropriate genre for CCS preparation. At some point, we need to articulate and publicize the literacy skills we teach. Assessment is one way to do that.
LikeLike
August 22, 2013 at 12:07 pm
Thanks, Anna. You are quite right to draw attention to the higher order literacy skills these databases cultivate. In terms of the significance of these databases to a program and by extension a university’s mission, several issues, often overlapping, are at play. For one, ECCO, Burney, and similar databases afford students opportunities to conduct original research and engage in more sophisticated, dynamic ways with the past and conduct more original research. Yet, the usefulness of these tools is not limited to specialized study of a particular period or century by any means. Rather they are also provide unusal hands-on training in developing advanced informational literacies. These are important life skills.
LikeLike
August 22, 2013 at 12:26 pm |
The short answer is that the state of Texas, for the time being, has announced its implacable opposition to the Common Core, for all the reasons you can imagine.
http://en.wikipedia.org/wiki/Common_Core_State_Standards_Initiative
I like your larger point about articulating in what ways historical literary studies imparts specific information literacies or really ways of thinking to students and practitioners. This is something Julie and I thought about quite a bit, since we quickly discovered that even the term “information literacy”–by treating IL as a mere, pre-disciplinary skill–represents a seriously reductive attitude towards crucial aspects of student and practitioners’ research. Moving from an assumption about the need for “multiple sources of information” to one about “multiple forms of information” on a particular topic is a big deal, and students in lit studies should become comfortable doing it.
Julie’s link about the revision of the ACRL guidelines points to a more dynamic conceptualization that they seem to be calling “transliteracy.” One way to begin to thinking about this, is to imagine how “English studies” (which might encompass literature, rhet/comp, Creative Writing, linguistics) might fit into the picture offered by this white paper referenced by the ACRL:
http://www.knightcomm.org/digital-and-media-literacy/learning-and-teaching-what-works/
There are aspects of this that we are probably uncomfortable with. But I think if the MLA dialogue is to function as dialogue, it will have to confront issues like this one about how we as a discipline define and therefore assess literacy (or literacies).
LikeLike
August 22, 2013 at 12:33 pm |
Thanks, Dave! And the MLA dialogue (see MLA Commons to join in) is itself a form of transliteracy as it acts to broker conversation and exchange across various constituencies within and beyond higher education.
LikeLike
August 23, 2013 at 8:02 am |
Dave and Julie: I admire the article you wrote and particularly appreciate your discussion of how library skills courses can compartmentalize and thereby minimize how many research skills students take away from the class.
Can you say a bit more about specific exercises you used to help students learn about electronic resources or special collections at the point of need?
LikeLike
August 23, 2013 at 12:10 pm |
Sure. For basic IL, we’ve developed a routine I do with all my classes. We have a room in the library equipped with laptops for every student. (That’s what we use in all my classes when we’re trying to teach databases, etc.) (If you don’t have such a room, you might just have a library session where everyone brings or shares a laptop) At that class session, the librarian (in this case Julie) introduces students to key databases and the catalog and some searching techniques, then immediately has student groups simultaneously conduct searches on a variety of databases related to the assignment at hand (usually an annotated bib). Julie and I walk around to suggest keyword selections that could broaden/narrow their searches. Then we have the groups report out on a result or two, and what they learned. We talk about the different content strengths of each database, and how to move from one to the next.
The next major moment is what we called our “one step further” assignment, described in the article. The point is that once they’ve done some descriptive work on a particular item in Special Collections, they go back into the database to do a bit of background work on that item. They do this a total of three times, and each iteration gets a bit more elaborate in their contextualization of an item described in Spec Colls. After learning how to do basic database work, the value of the one step more is really that they pick up on whatever appeals to them in a fairly random assortment of texts and genres, and then do a bit of digging in the library’s databases. Students basically learn that the point of research is to investigate what interests you, and I get much, much more interesting topics and questions after they’ve done this kind of work.
The final stage is the final research paper, which as I’ve mentioned requires a pairing of an assigned with an independently-chosen, non-canonical work, with a rationale for their pairing. I liked the results of this so much I began doing it in all my grad classes. All throughout I’m giving, and Julie is giving, as much support as we can for them to pursue their questions. They do get better at this as we go along, and it’s unlike any other classes they take.
LikeLike
August 23, 2013 at 1:35 pm |
Thanks, Dave, for this very fine, detailed look at your activities for this clearly wonderful class. I am sure students overall are so appreciative of the class and its format. I would also add that I see lessons/points of carryover here for gen ed research writing classes.
LikeLike
August 23, 2013 at 4:29 pm |
Thanks, Dave. This is very helpful in generating ideas for how to habituate students to these databases. I see the logic of using a computer classroom, where everyone has access to the databases. I’m going to have to think of some early targeted assignments to help advance that habituation.
LikeLike
August 24, 2013 at 12:12 pm |
One of the biggest insights we gained was that a looping, recursive yet building assignment using a resource, like our 3-time special collections to database assignment, was the best way to achieve transformational learning. This is one reason why so many of the one time “library day” style class visits fall short, no matter how well designed. it also means that you’re integrating the search, locate, and aggregate aspect of IL with your disciplinary content (in this case, the gothic novel), to make it reflect the level of independent work they need to do in a particular field. Finally, it means that you will not be the expert on everything they find or pursue, because there is an infinite horizon of possible questions, and so students find all sorts of new ways into the material, depending on their own background, interests, etc. I had one person who got interested in late 18c Druids; someone else who wondered about livestock husbandry in Jamaica; a third who was interested in Franklin’s economic theories, etc. etc.
LikeLike
August 24, 2013 at 12:20 pm |
I agree wholeheartedly, and I have constructed a multi-stage research assignment for many years now in my seminar and upper-level classes. As Dave notes, it enables students to engage with material in ways that truly interest them. It also makes the process more manageable for them as well as offering an immersive experience.
LikeLike