Delivering Aggregate Data: Supporting students while enabling institutions to act

A lot can be learned through site user and usage assessment. With a site like Callisto, where users are entering private and sensitive information, it becomes increasingly important that data is collected, analyzed, and shared carefully and intentionally. However, this doesn’t mean that there isn’t information to learn from – a lot can still be learned from user and usage patterns and trends.

There are two different types of data – individual data and aggregate data. Individual data, pretty obviously, is data about an individual user. This type of data discloses information about one singular user, and thus does not give privacy or anonymity to the user. Any piece of identifiable information could hypothetically be used to learn more about one single user. For something as sensitive as disclosing sexual assault, it is imperative that user data can not be tied back to a unique user. Aggregate data, however, is data that is combined to show the data of many users; typically used for reporting or analysis.

The other important thing around aggregate or individual data, specifically when the data relates to information about incidents of sexual assault, is around mandated response upon disclosure. If the school were able to tie an assault to an individual, whether it be a perpetrator or a survivor, they could be required to take prompt action under Title IX. With aggregate data, the school is able to identify trends and patterns without the obligation to take action.

Why is this important to Callisto? Aggregate data collection is extremely important to Callisto. From it we gather a few different types of information which have helped our campuses learn about their students, support survivors, and tailor their education and prevention work. This information includes the following:

  1. ASSAULT RATES: We collect information about how many students have created private records but chosen not to report, how many students have entered perpetrator identities into the matching system, and how many students have created accounts. These numbers are not available from any other form of reporting, because these relate to assaults that students are choosing to not report. We are able to give campuses a much greater depth of understanding around rates of sexual assault than only through reporting numbers.
  2. CALLISTO USERS: We ask students their class standing and their gender, with the knowledge that anonymous demographic data will be given to their school. Schools can also opt to have additional demographic questions included. The same questions are asked in regards to the perpetrator. Victimization and perpetration trends are nearly impossible to assess with reporting numbers averaging around 10%. Callisto can increase the school’s awareness of who is involved in sexual assaults.
  3. INCIDENT TRENDS: We collect anonymized data from Callisto’s multiple choice questions about when the assault happened, where it happened, who was involved, and what happened. We never provide information that could identify any individual student; we remain very serious about the privacy and safety of our users. However, through aggregate data reports, we are able to provide schools with valuable information like the percentage of assaults that included alcohol/drugs use and how students are identifying the non-consensual nature of incidents.
  4. USAGE PATTERNS: We collect information about what time, day, and week in the year students are accessing Callisto so that we can match this information up with a school’s academic calendar to better understand trends of when students need to access resources and have the ability to intentionally target education, prevention, and support services. We are able to know not only when students are reporting assaults, but also when they want to learn about support services, when they’re creating accounts, and when they’re accessing sections of the site with information about supporting friends. All of this information can help a school improve pre-existing on-campus efforts as well as develop new programs and prevention efforts.
  5. ASSESSMENT: We use Callisto site data to better understand how students use Callisto, when they leave the site, how long they stay on the site, what features they use  most, and other trends of their usage. While providing data to campuses is extremely important, so is our ability to improve Callisto to best meet the needs of the students it is serving.

What does Callisto do with data? Callisto compiles analyzed data reports and delivers them to their respective schools on a semi-annual basis so that schools can learn about assault rates on campus, how many people are using Callisto, when and how students are using Callisto, and victimization and perpetration trends. Because data collected by Callisto is always aggregated and anonymized, schools will never receive information that can be used to identify an individual user. By using a third-party platform like Callisto, schools are able to use data to their benefit while remaining in compliance with Clery reporting and FERPA regulations.

Data delivered to the schools is owned by the schools. Callisto aggregates data across schools to learn about nation-wide trends and patterns, but does not have ownership over individual school data. Callisto hopes to continue to provide research- and data-driven tools, but will not share individual school data without permission

What does the future hold? As Callisto continues to grow, we will be able to collect more and more aggregate data. This data will show national trends of when students are most likely to access resources around sexual assault, and can be matched up with days of the week, holidays, and other events that might impact the student experience. The data will also show the rate of students who want to process an assault but are still hesitant to report, and can help schools get a more accurate picture of the sexual assault climate.  Schools will have the ability to better target campus programming and to continue working to eliminate barriers to reporting.

Callisto will continue to help schools tailor their education, response, and prevention work on campus by providing robust aggregate data. As the reach of Callisto expands, so will the message that committing sexual assault holds serious repercussions both on campus and off. Callisto fights back against a society where only 1% of perpetrators is held accountable and provides a real deterrent for committing sexual assault.

Callisto: Technology That Isn’t Trendy

In an age where it seems like our cell phones can do pretty much anything for us, it only makes sense that sexual assault awareness, prevention, and response work infiltrates the tech world as well. It seems like every day there is a new app created that is intended to combat sexual assault. Mobile applications are trendy. However, when developing Callisto, it became increasingly clear that a recording and reporting tool like Callisto needed to be a website and not an app. Here are some of the reasons why.

It is unlikely that students will preemptively download an app before an assault with the assumption that they will need it later.

Applications can only be used if they are downloaded onto the phone. Because the moments after an assault are highly traumatic for many survivors, it is not likely that students would find and download the app before recording what happened. Data from the Association of American Universities Climate Survey on Sexual Assault and Sexual Misconduct showed that over 50% of students thought themselves not at all likely to be victims to sexual assault, which means that very few students would preemptively download the app with the expectation of needing to use it at a later date.

Downloading an app means having the icon and name on your phone where others can see it.

Survivors we spoke with during our formative research for Callisto were very cautious about protecting their status as a survivor until they were ready (if ever) to share with others. They were also very protective of their story through privacy locking documents or tearing up what they wrote. The fear of having someone discover this information was amplified by the idea of a mobile app.

Students are more likely to record details of an assault in the privacy of their apartment or dorm room, not out in public.

Through countless interviews with survivors, we discovered that detailing an assault would not be something that they would likely do on a bus, walking around campus, or at an event. Our formative research informed our decision to make Callisto something that would be most easily accessible in the privacy of someone’s own room.

Students need a centralized, easily accessible resource for all information around reporting information and options.

Based on the AAU Survey results, only about 25% of students are knowledgeable about where to make a report about sexual assault or sexual misconduct at their school. The school-specific Callisto site can be linked to directly on any already existing school website where students may go to find resources. This allows for easy navigation from sites students may already visit such as the counseling center, the campus police, or the Dean’s office so that information that may have previously been separate is now in one consistent location. Navigation between websites and phone apps isn’t as seamless as site to site.

Websites can be mobile optimized so that they can easily be used on cell phones and tablets.

Mobile applications are often not able to be used on computers. Callisto is mobile optimized and is easily navigated on all devices. It isn’t required for users to download something on to their phone to use Callisto on their phone. Because Callisto is a mobile-optimized website, it can do everything that a phone app could do along with much more.

None of these reasons are to say that mobile applications are not useful or beneficial. There are many great resources out there that are apps, and we support all of our partners in the work to combat sexual assault. For us, building a website and not an app was an important and intentional decision.

Are you going to be at ACPA or NASPA this March? Come visit Ashley & Kate to learn more about Callisto or go through a demo of the site! Email us at [email protected]

Do Survivors Want Callisto?

Results of a Conceptual Acceptability Survey from November 2013

Background & Overview

In November 2013, we were still deciding whether or not to create Callisto, our sexual assault reporting system. We thought it was a brilliant idea, but then we would, because it was our idea. Before we started seriously fundraising, looking for customers, or spending significant time on the concept, we wanted to check: do survivors even want Callisto?

So we conducted something called a “conceptual acceptability survey” – meaning a survey to assess whether or not your target population even likes an idea. We also wanted to measure “retrospective likelihood of use” – meaning how likely people think they would have been to use the system, had it existed when they needed it.

Here’s the overview: We recruited 162 18-30 year olds through a Facebook event invite. A third had been assaulted while in college or grad school and 72% had never reported. The vast majority of survivors (98%) thought Callisto was a good or great idea. 44% of survivors thought they would have “definitely” or “probably” used Callisto if it had existed, and 40% would have maybe used it.

Methods

In November 2013, we recruited a convenience sample of young adults  to take an online survey via a Facebook event invite. We invited friends. Friends invited their friends or sent the survey link out. You could call it snowball sampling but it was less deliberate than that.

At the time, Callisto wasn’t even called Callisto yet – it was SHARP for the “sexual harassment and assault reporting project.” Because who doesn’t like acronyms?

Here’s what the Facebook event looked like:

Image of facebook survey event invite

And here’s the first page of the survey:

Page 1 SHARP survey

We asked a mix of multiple choice and free-response questions. We asked for consent to use quotes from the free-response answers. You can see some of those quotes on the Project Callisto website. Here’s how we asked whether we could quote people:

consent to quote

Demographics of participants (n=162)

We had 162 young adults, ages 18-30, take the survey. Most (68%) were cisgender women. We think most were from the U.S. but we didn’t ask. The median age was 27 and the mean age was 26.

sex questionsex answer

 

gender questiongender answer

“Other” in the above pie chart includes transgender, gender transgressive, genderqueer, and other genders.

 

age question

age answer

History of participants

A quarter (26%) of participants had been sexually assaulted while in college or grad school, and 8% were not sure whether they had been sexually assaulted. We did not define sexual assault for participants. More women than men had been sexually assaulted.

sexual assault questionsexual assault answer

females sexual assault answermales sexual assault answer

Most (72%) people who had been assaulted/possibly assaulted hadn’t reported. Those who said they had been assaulted were more likely to have reported than those who weren’t sure whether they had been assaulted or not (which makes sense).

reporting question

reporting answer

reporting answer among yes assaultedreporting answer among not sure assaulted

Most people (75%) had a friend who had been sexually assaulted or harassed while in college or graduate school.

friend assaulted questionfriend assaulted answer

Conceptual acceptability

And now to the meat of it all: did people like our idea? Answer: yes.

Women liked the idea more than men, survivors of sexual assault liked it more than people who hadn’t been assaulted, and people who had friends who had been assaulted or harassed liked it more than those who didn’t.

the idea

conceptual acceptability question

acceptability answer acceptabillity answer - womenacceptabillity answer - men

acceptabillity answer - assaultedacceptability answer - not assaulted

acceptability answer - friend assaultedacceptabillity answer - friend not assaulted

When we asked for more thoughts on the idea, we got some great responses:

“It should include information on how to contact a counselor or therapist. Additionally, if there are other people serving as advocates for victims it could include their information as well. Contacting the police is a daunting task, but having someone by your side through the process would be comforting.” – Female, 29

“Any reporting system that creates an objective record that can’t be disregarded by the school seems like an amazing idea.” – Male, 27

“This needs to be open to all college community members – staff, professors, etc. to file complaints. This is amazing!” – Female, 20

“This seems particularly important for identifying repeat-offender perpetrators, whose victims otherwise might think they were the only ones. Generally, this seems like a hugely important way to increase agency for assaulted students.” – Female, 26

Retrospective likelihood of use

Do survivors think they would have used Callisto to report or save a record if it had been an option? Yes and maybe. Those who had reported their assault would have been more likely to use Callisto  (a lot of “yes”s) than those who had never reported (a lot of “maybe”s).

We asked this question to both people who answered “yes” or “not sure” to the question about whether they had been assaulted in college/grad school. Those who had answered “yes” were more likely to say they would have used Callisto than those who weren’t sure whether they were assaulted.

likelihood of use

use answer

use answer - reporteduse answer - didn't report

use - yes assaulteduse - not sure assaulted

When we asked why people think they wouldn’t have used Callisto, the main reasons were:

  • They felt confused about whether what happened to them was really assault

    “In my case I feel it’s debatable whether it was assault or not so I might not put it in the system. If I felt like I was definitely assaulted I probably would use this proposed system” – Female, 26

  • They felt confused or intimidated by the reporting system at their school

    “It seems a bit complicated at this point in time. and I already had a problem with the complicated reporting system at my university” – Female, 22

  • They felt ashamed

    “There was a lot of shame around what happened.” – Female, 28

  • Drugs or alcohol were involved

    “Each time I was harassed/assaulted, I was under the influence of drugs, alcohol, or both, and while they may have impaired my ability to keep myself safe, I don’t deserve what happened to me. That said, many college administrators/police officers/lawyers don’t agree.” –Female, 21

The main reason survivors would have used Callisto was because:

  • It would have been helpful just to write it all down as a first step or as a therapeutic tool

    “It was hard to imagine coming forward in an immediately public way, but to start by chronicling what had occurred would have been a helpful and important first step.” – Male, 29

  • The ability to know about repeat offenders

    “The ability to create a record, to have control over whether or not the info is disseminated while at the same time contributing to a database that will be attuned to repeat offenders — this is extremely desirable.” – Female, 24

  • It is not as intimidating as other forms of reporting

    “because it is not as scary as filing a police report. i was not 100% sure that what happened to me was a crime at the time, so that seemed intimidating and possibly ‘excessive.’ But I would have like to file something, to see if he had done this before and just to validate my experience” – Female, 25

Of note, we did not ask about likelihood of use of the matching escrow feature in Callisto – the ability for survivors to report only if someone else reports the same assailant.

Conclusion

These results gave us faith that a lot of people, especially survivors, thought Callisto was a good idea. Not every survivor would use it, and the feedback we got helped us to identify what the major barriers to adoption might be (shame, difficulty with labeling what happened, lack of transparency in reporting procedures, use of drugs of alcohol). We have been working to address these barriers in the design and marketing of Callisto (more on that in a later post).

In December 2013, we started conducting focus groups with college survivors. In January 2014, we started meeting with schools about the concept. In October 2014, we launched our first crowdfunding campaign for Callisto. In August 2015, we launched on two pilot campuses (USF and Pomona College). Throughout all this, we continued to engage survivors through focus groups and one-on-one interviews. This survey is where our research into Callisto started and that research continues today.

Special thanks to Natalie Price for her assistance in preparing this post.