Jenna Tingum
Idle

Synopsis

Through the Data x Power fellowship, this research and model was developed to solve an issue within the progressive tech space - how can we easily group conversations for follow-up without adding extra steps? Within this project’s development, a broader question arose around progressive tech more generally and the potential of AI usage within progressive spaces. This project, then, serves as a model for how organizers and technologists can develop tech together to ensure greater buy-in and longer-term use.

DxP

For several years, organizations across the progressive movement have been experimenting with, frustrated by, and cementing a deeper understanding of data and technology to support their work. Although Data x Power members range in size and scope, we share common problems in tech infrastructure and implementation, as well as navigating privacy, security, and data ethics.
In 2021, Ford Foundation partnered with re:power Fund to create a space to build collective strategy and innovation through a 10-month fellowship for movement-centered data experts. Each year, a new cohort of twelve fellows are selected to work in pairs on a project that will address network-wide data and technology issues. They will undergo a series of skill-based trainings, receive a mentor, and have access to funding that enables exploration, experimentation and completion of the project. By pouring resources into individuals, we hope this fellowship serves as an incubation space for collective learning and solutions that can serve the larger movement.
Through the Data x Power fellowship, this research and model was developed to solve an issue within the progressive tech space - how can we easily group conversations for follow-up without adding extra steps? Within this project’s development, a broader question arose around progressive tech more generally and the potential of AI usage within progressive spaces. This project, then, serves as a model for how organizers and technologists can develop tech together to ensure greater buy-in and longer-term use.


About This Project

The Problem

When organizers are talking to folks in their communities, they often take notes about their interactions to document and aid in follow-up. But many tools don’t allow for easy search terms to group these folks by their key issue areas. For example, if a community member is talking to an organizer about the affordability of their prescriptions and healthcare generally during this current administration, how can we quickly tag this person as caring about “healthcare” for future follow-up in upcoming program?

The Process

Many CRM tools already exist to house organizer notes, so the solution will not be developing a new tool. Instead, the solution here will be presenting the research and proven methods to implement this into existing tools. Knowing that this solution will require use of Artificial Intelligence and, through personal experience, that AI is a divisive topic among progressives, an additional point of research was added to understand how implementation of a tool like this would work. Ie. would organizers be too skeptical of AI to establish enough buy-in for this to work?

The Solution

Presented on this site is a comparative analysis of different methods that could create tags of organizer notes and a demo of one of the methods. The tag examples of constituent issues can be found here as well as some the full list of sample notes used to test the models. Additionally, thoughts on AI implementation among progressive organizers are presented, in the form of anecdotal survey responses.

Project Demo

Here are some sample field notes to try out, but feel free to test your own!

  • Talked to Lou at the door. He supports our candidate for state house but it’s clear his top issue is affordability. Talked about grocery and gas prices rising plus childcare costs with a baby on the way. Gave some resources on local childcare and charities with lower-cost supplies.
  • Diabetic, insulin costs rising is taking a toll on her and her family. Interested in our international cuisines event so left some information with her and will follow-up.
  • Talked to him on Monday 3/16 morning, isn’t affected by the recent rent hikes in the area since he owns his condo. He says his main issue is the 2nd amendment and the worry that progressive leadership might take away his guns. Feels strongly about gun ownership, big NRA member
  • Has a lot of friends involved in Planned Parenthood work. Interested in getting involved in ballot initiative work for this year. Maybe can connect us with her network.
  • Has a 7 and 9 year old and thinking about moving schools because they’re worried about the quality of education at the public school. Lots of religion being pushed that makes them uncomfortable. Was asking about our candidate’s stance on school choice and resources about vouchers.
  • Talked to Christine on Saturday afternoon. Interested in volunteering with us. Cares a lot about sustainability and wants to learn about composting. Needs more voter education on voting by mail.
Block 1
Block 1 Description
Ctrl+Enter to run


Evaluation Metrics

Further details about how to determine which model will work best.

Model Considerations

supervised vs. unsupervised training

AI Reception

Plug for Aaliyah's project, organizer sentiment, data security + privacy, governance

Cost and Maintenance

Cost and Longevity

Project Details

Introduction

Problem

Progressive organizers go out into the field day after day and have deep conversations with the community. They take diligent notes on their conversations for follow-up, but how can we easily group these conversation notes for follow-up without adding extra steps? How can we automatically turn this qualitative data, that folks are already collecting, into categories defining the issues community members care most about?

The answer is with an NLP model integration that takes field notes and converts them into issue tags. This answer, though, poses a larger question to the progressive community - how do we safely and effectively integrate AI tools into our ecosystem? How do we do so safely and as good stewards of community data? How can progressive tech developers involve organizers in their development process to ensure organizer buy-in, especially when distrust of AI runs deep in the progressive community?

Instead of developing an entirely new tool, I’ve detailed my research and thought process below so that existing field tools can implement a similar model into their systems. Each avenue I explored has its own demo to try out the model’s performance followed by some comparative analysis.

Here is a list of all the tags chosen for this project, using this Gallup Poll’s list of most important issues influencing the 2024 election as a starting point.
  • Economy
  • Democracy
  • Terrorism
  • Immigration
  • Education
  • Healthcare
  • Gun Policy
  • Abortion
  • Taxes
  • Crime
  • Foreign Affairs
  • Energy Policy
  • Race Relations
  • LGBTQ+ Rights
  • Housing

Keyword Extraction (KeyBERT)

What Is Keyword Extraction?

The first and simplest approach to this problem is keyword extraction. It requires the least amount of processing power and coding effort, but isn’t very intelligent - needing the exact keyword to be present in the note text to match the list of keywords. Here are two short and relatively easy examples to demonstrate the model’s pros + cons. .
KeyBERT Demo Keyword Extraction
Description
Ctrl+Enter to run

Examples

“Talked to Lou at the door. He supports our candidate for state house but it’s clear his top issue is affordability. Talked about grocery and gas prices rising plus childcare costs with a baby on the way. Gave some resources on local childcare and charities with lower-cost supplies.”
It’s clear the main tag here should be “economy”, but the model turns up nothing. Now, try changing the word “affordability” to “the economy” and it quickly returns the correct value.

Pros & Cons

Pros

  • Simple Implementation
  • Free

Cons

  • Can’t interpret note as a whole thought, only searches for a word
  • Not a real option for nuanced organizer conversations

Pre-existing LLM (Claude)

On the opposite end of this model spectrum is a fully developed LLM that’s pre-existing and pre-packaged. This is a far more intelligent approach but requires a lot more computing power, cost, and energy. It’s also likely to have a steeper buy-in process with organizers.
My colleague Aaliyah Wood conducted a survey to gather some anecdotal feedback from organizers on using AI for field work. Here are some of their thoughts:
  • It can definitely be useful and I do personally use it sometimes, but the litany of moral issues around it (impact on the workforce, slop, intellectual property issues, energy and environmental impacts, data privacy issues, etc.) make me feel generally pretty negative about it.
  • From what I know about data centers and how harmful they are to the surrounding community, how much water they require, the data privacy issues around it etc, I don’t have the best opinion about it.
  • Bad for the environment, can stunt human learning, needs extensive human oversight. Unsure of its net positives on the world.

Aside from organizer buy-in, AI companies generally have different, more corporate priorities that often lead to differing ethical considerations than those of the progressive movement. For example, ChatGPT, another leading LLM, recently struck a deal with the Trump administration’s Pentagon to provide them with their data and tools. Claude was chosen in this case because of its established use cases in the progressive community (for example here), its more neutral brand perception, and its refusal to adhere to these same Trump administration asks that ChatGPT did.
Ultimately, though, corporate LLMs remain ethically ambiguous and ever-changing - privacy, governance, and corporate social responsibility must be considered.


Unsurprisingly, an LLM like Claude can create correct output to simple organizer notes. Try this example to test it: “Has a 7 and 9 year old and thinking about moving schools because they’re worried about the quality of education at the public school. Lots of religion being pushed that makes them uncomfortable. Was asking about our candidate’s stance on school choice and resources about vouchers.”
Claude Demo LLM
Descriptionss
Ctrl+Enter to run

Pros & Cons

Pros

  • Simple to implement
  • Stronger understanding of nuance and overall themes
  • Can attempt to handle spanglish or english misspellings

Cons

  • Costly, especially to ensure data privacy
  • Unknown Environmental Impact
  • Steeper organizer buy-in

Unsupervised Machine Learning / NLP

Spacy

Ctrl+Enter to run

Pros & Cons

Pros

  • Good
  • Not bad

Cons

  • Not as good as could be

Zero Shot Classification

Ctrl+Enter to run

Pros & Cons

Pros

  • Good
  • Not bad

Cons

  • Not as good as could be

Sentence Transformers

Ctrl+Enter to run

Pros & Cons

Pros

  • Good
  • Not bad

Cons

  • Not as good as could be
🐙 GitHub

Contact

More words

✉️
Email
🐙
GitHub
💼
LinkedIn

Acknowledgements

⭐
Aaliyah Wood
1
⭐
Kate Beeken
2
⭐
Sam LeBlanc
3
⭐
LJ Oks
4