Based on a presentation titled “Navigating Generative AI: Early Findings and Implications for Research, Teaching, and Learning” given at the Coalition for Networked Information (CNI) on March 26, 2024. The presentation is available in full here.

The researcher prompts the AI-powered research tool with the phrase, "Find some related topics." The tool generates a list of key topics in the document.
The researcher adds this prompt, "What is that based on?" in response to results generated by the AI-powered research tool. The tool replies, "According to this article it is..."

Advances in technology have always been an accelerator of ITHAKA’s mission to improve access to knowledge and education, and the advent of AI is no different. With the release of JSTOR’s AI-powered research tool in beta, we are delivering on our promise to collaborate with our community to use AI responsibly and transparently in order to empower researchers and learners to expand and deepen their work.

As of this writing, 22,234 active users at 5,437 institutions in 149 countries have used the tool since its launch in August 2023. We are actively learning from usage, real-world applications, and user feedback in a limited beta environment to build out capabilities thoughtfully, intentionally, incrementally, and most importantly–based on evidence. This blog post delves into the rationale, construction, and current functionality of JSTOR’s interactie research tool (beta), as well as the key themes and insights that have emerged from our analysis of its usage.

569,846 tool actions from 22,234 active beta users at 5,437 institutions in 149 countries (as of June 18, 2024)

As of June 18, 2024 (click to enlarge)

The development of JSTOR’s interactive research tool (beta)

Our approach

The massive user adoption of ChatGPT in late 2022 signaled a significant shift on the horizon in user behavior and expectations. People were drawn to the idea of conversational interaction made possible by AI and other advanced technologies. We recognized the transformative potential of this shift and wanted to understand its impact on the scholarly community, and to explore how it might be used responsibly and thoughtfully to advance our mission of making knowledge more accessible. The world was changing quickly, and we knew we needed to act fast to help our community navigate this transformation.

To start, in spring 2023, Ithaka S+R, a strategy and research consultancy from ITHAKA and peer service to JSTOR, announced a multi-year collaborative research project focused on helping colleges and universities understand the impact of these advanced technologies and chart a productive path for their use in higher education. Around the same time, on JSTOR, we kicked off an ambitious project to leverage AI and leading-edge technologies to bring a research tool to the platform in beta by the fall of 2023.

The rationale behind developing JSTOR’s interactive research tool (beta) is rooted in a commitment to democratize research, enhance educational experiences, and support scholarly endeavors across various academic levels. The guiding principles of this initiative include using AI and other advanced technologies in ways that uphold JSTOR’s core values of equitable access and scholarly integrity; support education by enhancing user experience without replacing the educational process; help the academic community make the best use of these capabilities; and share what we learn throughout. We opted for a limited beta approach to maximize learning and iterate in a low-risk environment.

Building the tool

The development of JSTOR’s interactive research tool involved a rapid and intensive process aimed at quickly delivering a functional beta to users to kickstart the learning process. Driven by a team of engineers, data scientists, designers, and researchers, the initial phase involved meticulous planning to identify the core user problems that this technology could uniquely solve for JSTOR’s users. Following this, the team embarked on a rapid development cycle, quickly building the tool to ensure a swift beta launch. This agile approach was complemented by a user-centric design methodology, where feedback from users was incorporated even before the beta release to refine the tool’s functionalities and ensure it met user needs effectively.

The technological backbone of the tool relies on advanced AI models, primarily a combination of OpenAI’s GPT-3.5 and smaller open-source models. At the time that we were building the first version in the summer of 2023, these models were chosen for being robust and able to handle various text analysis tasks. Those choices allowed us to move quickly while actively exploring and leaving the door open to alternatives. As new LLMs are created, we have the ability to substitute different models based on the mix of variables, such as accuracy, cost, and capacity. The tool employs sophisticated vectorization techniques, which transform questions and text into vectors to enable precise semantic searches within documents. Custom prompts are another crucial aspect, carefully crafted to ensure that the technology generates accurate and relevant responses that enhance the overall user experience.

Current functionality

In its current form, JSTOR’s interactive research tool is designed to provide users with a range of powerful functionalities for interacting with the full text of a single item, including:

  1. Custom summaries: The tool can provide a summary directly related to the user’s search query, as well as a general summary of the document, allowing users to quickly grasp the content and relevance of academic texts.
  2. Recommended topics: The tool can recommend related topics based on the document’s content, enabling users to explore additional research paths and deepen their understanding of the subject matter.
  3. Related content: The tool can display documents that are conceptually similar to the current text being viewed, helping users to find additional relevant resources quickly and efficiently.
  4. Question and answer (Q&A): Users can ask specific questions about the document, and the tool generates answers using the document’s content to ensure accuracy and relevance. This feature is particularly useful for users seeking detailed information on specific aspects of a document, including topics and concepts.
How the Q&A feature works: 1. User types a question: this can be a keyword, a phrase, a question; 2. Question is vectorized to represent the meaning of the question; 3. Vector is used to find relevant portion of text so that it can be used to answer the question; 4. Question, text, and prompt is sent to LLM–prompt includes instructions including: provide concise answers, use only the document text and nothing else, don’t add any details/tips if not mentioned in the text; 5. LLM generates a response, per instructions; 6. Response is presented to the user

How the Q&A feature works (click to enlarge)

We encourage users to provide feedback on the performance of the tool, with options to rate the quality of responses and provide open commentary about their experience. This feedback helps us make continuous improvements.The tool also maintains transparency by linking back to the original text sections used to generate answers, ensuring users can verify the information provided (see screenshot below). We’ve also engaged subject matter experts from a range of academic disciplines to conduct in-depth, ongoing evaluations of the tool’s output to further ensure accuracy.

A split-screen image showing a page from a book on the left and an interactive AI-powered text analysis tool on the right. The book page discusses Gothic novels and their elements. The text analysis tool, titled

The tool maintains transparency by linking back to the original text sections used to generate answers, ensuring users can verify the information provided (click to enlarge)

Key learnings

Improved research efficiency

The tool has been praised for its ability to streamline the research process.

A faculty member noted, “I can do in a day what it used to take me four or five days… AI isn’t doing any of the intellectual work for me… It’s not thinking things for me; it’s showing me where to look.”

This highlights how the tool aids researchers in quickly finding relevant materials, allowing them to focus on deeper analysis and comprehension. Additionally, users are spending significantly more time on JSTOR with the AI tool, with session durations and page visits more than doubling compared to non-beta users. This heightened engagement indicates that the tool is making the research process more interactive and appealing.

Another user shared, “I find that I’m spending more time capturing and downloading the AI summaries than downloading PDFs… I will then download a few of the PDFs that seem appealing.”

This feedback underscores how the tool is changing the way users interact with academic content, making it more efficient and effective. Researchers can spend more time engaging deeply with the most relevant and salient texts to their inquiry, rather than wading through the sometimes overwhelming number of search results to identify them in the first place. The tool also keeps users from having to spend large amounts of time reading articles only to find that they are not relevant to their research. This is particularly valuable for the many articles in the humanities and social sciences that do not include abstracts.

Enhanced engagement and interaction

Distribution of tool usage from launch in August 2023 to the present, including Q&A (dark blue), custom summaries (red), related content (orange), and recommended topics (teal), as of June 18, 2024 (click to enlarge)

Evidence of enhanced engagement and interaction is readily apparent from the distribution of tool usage illustrated above. Of the four main features of the tool—summary, topics, related content, and Q&A—summary and Q&A are used more frequently, with the Q&A feature becoming the most highly used over time. This tells us that while users are engaging with all of the capabilities, they are primarily drawn to the conversational aspect the Q&A feature delivers. In this way, they can “talk to the document” and engage in a kind of “conversation” with the text. Moreover, they can trust and verify that the responses the tool provides are credible and accurate by clicking on the footnotes pointing back to the original sections of the text from which the answer was generated.

Because this initial evaluation takes place between users and a non-sentient research tool, there are no “dumb questions.” There’s also no limit to the number of questions users can ask, the outputs are easy to read and understand, and the responses are provided immediately. We believe that the transparency and traceability created by these guardrails relieve some of the uncertainty and doubt researchers may feel when beginning new research, thus freeing them up to take more creative risks as they explore innovative lines of inquiry while remaining grounded in the literature.

Accessibility and the democratization of research

JSTOR’s interactive research tool caters to a wide range of researchers, from novice students to experienced scholars. This versatility makes it a valuable resource across different academic disciplines and levels of expertise.

For novice researchers, the tool helps in finding and evaluating materials for relevance and understanding complex academic texts. This is particularly helpful for high school and undergraduate students who are still developing effective research skills and can benefit from the tool’s ability to make scholarly content more accessible. The tool prepares them to engage more deeply with the material, much like knowing the plot of a Shakespeare play before reading it enhances comprehension. Helping early career researchers find the “right” material faster and facilitating understanding about why it’s relevant and useful to them reduces frustration, wasted time, and energy that’s better spent exploring the text and developing and testing hypotheses against what they’ve found.

A student volunteered, “Honestly, time is everything… Wasted time and difficulty levels will go down significantly because of this. I mean, again, JSTOR boasts that we have 1,000s of articles, which is awesome; I love JSTOR. But sifting through those articles is kind of hellish.”

These early research experiences are particularly significant because they play a critical role in shaping student attitudes and perceptions of their own abilities and interests related to academic research. By reducing logistical friction while students are being exposed to research methodologies and academic writing—possibly for the first time—JSTOR’s new tool can lower the barrier to entry and allow students to learn and develop their skills unimpeded, iteratively, and unselfconsciously. Given the nature of the interchange that takes place between the user and the tool, as well as between the user and their instructor and classmates, the tool allows them to collectively master research skills that will benefit them in their academic careers and beyond.

A librarian remarked, “It would make it easier for them [students] to use the database and get information out of it, lowers the scariness factor in the database, it can be frustrating when you’re searching, lowering the annoyance factor as well.”

Faculty, PhD students, and other more advanced researchers use the tool to explore complex conceptual relationships across disciplines and content types. It’s a generative tool with the potential to supplant bandwidth-sapping tasks so researchers can do the deep work of understanding the material, forming hypotheses, and building a point of view that ultimately expands the collective academic perspective and discourse on a given topic. This democratizes research by providing support that is often only available to those with dedicated research assistants. By providing essential research assistance, JSTOR’s generative AI tool has the potential to level the playing field for faculty, researchers, and students at institutions with limited resources. In turn, the democratization of research can foster a more inclusive academic environment.

Educational applications and implications for deeper learning

Instructors find the tool valuable for supporting both the execution of student research and the development of critical research skills.

Another librarian observed that “It makes research interactive, which is great… I think it would change how I teach, making it a little less introductory…”

Not only does the tool help instructors teach students how to find and evaluate research material, but it also allows students to self-serve, move beyond rudimentary barriers in the research process, and delve deeper into their arguments, interpretations, and evidence. In this way, the tool enables educators to provide more personalized learning experiences adapted to the unique needs, preferences, and interests of each student, an increasingly difficult task given their growing responsibilities and expectations.

More generally, the tool also supports the development of dynamic and responsive curricula, allowing instructors to curate reading lists, develop more relevant and engaging course materials and assignments, and perhaps even offer new perspectives on traditional subjects, sparking more captivating and impactful discussions in the classroom.

Future developments and enhancements

JSTOR’s interactive research tool is poised to transform academic research, teaching, and learning by providing innovative features that enhance efficiency, user engagement, accessibility, and even the impact of teaching and research as a whole.

So what’s next for AI and other advanced technologies on JSTOR? The tool’s potential for translation and adaptability for different reading levels means that more users worldwide can access and understand academic content. This not only promotes global knowledge-sharing, but also supports users with different learning needs and preferences. Additionally, the tool’s potential to generate summaries and other content in multiple formats, such as audio and video, could significantly enhance accessibility for users with disabilities or those with different learning styles and preferences. JSTOR plans to expand the tool’s capabilities to cover broader document scopes, such as entire books and other content types, which will further enhance its utility. We are also actively exploring the possibility of extending the tool into more areas of the platform and research workflow. Continuous user feedback and iterative development will ensure the tool evolves to meet the real-time and ever-changing needs of the academic community.

Have you tried JSTOR’s interactive research tool? Share your experiences and thoughts with us. How do you think AI and other advanced technologies can further enhance academic research, teaching, and learning? We look forward to continuing the conversation and sharing how your feedback is directly influencing the evolution of leading-edge technologies on JSTOR.

If you’re interested in exploring the research tool, you can join the waitlist for the beta here.

Related content

About the authors

Kevin is president of ITHAKA, the not-for-profit that provides innovative services that benefit the academic community: JSTOR, Portico, and Ithaka S+R. He is an executive and entrepreneur with expertise in high technology and not-for-profit management. Kevin was the founding president of JSTOR (1995) and Ithaka (2004); the organizations merged in 2009, becoming known as ITHAKA and continuing under his leadership. Today, Kevin leads the ITHAKA staff of more than 300 people across four locations, empowering a diverse set of teams from technologists and designers to content experts and researchers.

Beth LaPensee is a Senior Product Manager at ITHAKA. Leveraging her MLIS and user experience background, Beth fosters a discovery-driven approach leading a team of engineers, data scientists, designers, and researchers to optimize content discovery and research on JSTOR. Recognizing the potential of generative AI, Beth spearheaded the development of an AI research tool.