What is campus climate?
Why is a positive climate important?
Why did Ithaca College conduct a climate survey?
Who conducted the survey?
Why was a non-Ithaca College researcher selected for the project?
How were the questions developed?
Why did some demographic questions contain a very large number of response options?
What was the Institutional Review Board (IRB) process for this study?
What will be done with data from the results?
What was the response rate goal?
What was the response rate?
How was respondents' confidentiality protected?
What is included in the final summary reports?
What protections are in place for storage of sensitive data, including for future secondary use?
Why was this a population survey and not a sample survey?
What was the survey timeline?
The current attitudes, behaviors, standards and practices of employees and students of an institution, often shaped through personal experiences, perceptions and institutional efforts.
Positive personal experiences with campus climate and positive perceptions of campus climate generally equate to successful outcomes that include positive educational experiences and healthy identity development for students, productivity and sense of value for faculty and staff, and overall well-being for all.
The idea to conduct a climate survey originated from interested students, faculty and staff who believed data from such a survey might be useful in improving the Ithaca College climate.
The Climate Study Working Group (CSWG), which included a cross section of students, faculty and staff, was charged with conducting Ithaca College’s climate survey. After a review of potential vendors, the group selected Rankin & Associates Consulting to conduct the survey. Rankin & Associates reported directly to the them. Although the CSWG regularly updated the Ithaca College community about its progress, the group — in consultation with Rankin & Associates — was solely responsible for the development, implementation and interpretation of the survey and its results. Susan Rankin was the consultant working directly with us on this project.
Rankin is an emeritus faculty member of Education Policy Studies and College Student Affairs at The Pennsylvania State University and a senior research associate in the Center for the Study of Higher Education. She has extensive experience in institutional climate assessment and institutional climate transformation based on data-driven action and strategic planning. Rankin has conducted multi-location institutional climate studies at more than 150 institutions across the country. She developed and utilizes the Transformational Tapestry model as a research design for campus climate studies. The model is a comprehensive, five-phase strategic model of assessment, planning and intervention designed to assist campus communities in conducting inclusive assessments of their institutional climate to better understand the challenges facing their respective communities.
In reviewing efforts by other colleges and universities to conduct comprehensive climate studies, several best practices were identified. One was the need for external expertise in survey administration. A survey relating to a very sensitive subject like campus climate is likely to yield higher response rates and provide more credible findings if led by an independent, outside agency. Members of a college community may feel particularly inhibited to respond honestly to a survey administered by their own institution for fear of retaliation.
The consultant has administered climate assessments to more than 170 institutions across the nation and developed a repository of tested questions. To assist in contextualizing the survey for Ithaca College, and to capitalize on the many assessment efforts already undertaken, the Climate Study Working Group (CSWG) was formed and consisted of faculty, staff and student representatives from various constituent groups at Ithaca College. The committee was responsible for developing the survey questions. The team reviewed selected survey questions from the consultant’s tested collection, and also included Ithaca College-specific questions which were informed by focus group results.
It is important in campus climate research for survey participants to “see” themselves in response choices to prevent “othering” an individual or an individual’s characteristics. Some researchers maintain that assigning someone to the status of “other” is a form of marginalization and should be minimized, particularly in campus climate research which has an intended purpose of inclusiveness. Along these lines, survey respondents in IC's survey saw a long list of possible choices for many demographic questions. However, it is reasonably impossible to include every possible choice to every question, but the goal was to reduce the number of respondents who must choose “other.”
An IRB application was submitted for the project and the IRB committee approve it: IRB 0616-06
All stakeholders — students, faculty, and staff — were invited to participate in the development of post-survey action initiatives. Four key areas were identified from the results:
- Fostering a deeper, more meaningful sense of belonging among students
- Improved career opportunities for staff
- Richer mentoring for faculty
- More transparent and inclusive decision-making at the administrative level
Target participation in the survey was 100%. Every response matters in surveys such as this, and all information provided is valuable in providing the most beneficial feedback and results.
Forty-six (46) percent of the campus community responded.
Of all respondents: 70 percent (2,672 individuals) were
undergraduates; 4 percent (157) were graduate students; 12 percent
(466) were faculty; and 14 percent (528) were staff.
Confidentiality is vital to the success of campus climate research, particularly as sensitive and personal topics are discussed. While the survey could not guarantee complete confidentiality because of the nature of multiple demographic questions, the consultant took multiple precautionary measures to enhance individual confidentiality and the de-identification of data. No data already protected through regulation or policy (e.g., Social Security number, campus identification number, medical information) was obtained through the survey. No personally identifiable information was shared in any publication or presentation resulting from the assessment.
Confidentiality in participation was maintained to the highest degree permitted by the technology used (e.g., IP addresses were stripped when the survey was submitted). No guarantees can be made regarding the interception of data sent via the Internet by any third parties; however, to avoid interception of data, the survey was run on a firewalled web server with forced 256-bit SSL security. In addition, the consultant and college did not report any group data for groups of fewer than five individuals, because those “small cell sizes” may be small enough to compromise confidentiality. Instead, the consultant and college combined the groups or took other measures to eliminate any potential for demographic information to be identifiable. Additionally, any comments submitted in response to the survey were separated at the time of submission to the consultant so they could not attributed to any individual demographic characteristics. Identifiable information submitted in qualitative comments was redacted and the college only received these redacted comments.
Participation in the survey was completely voluntary, and participants do not have to answer any question— except the first positioning question (staff, faculty) —and could skip any other questions they consider to be uncomfortable. Paper and pencil surveys were also available, and were sent directly to the consultant.
Information in the introductory section of the survey described the manner in which confidentiality was guaranteed, and additional communication to participants provided expanded information on the nature of confidentiality, possible threats to confidentiality and procedures developed to ensure de-identification of data.
The consultant provided a final report that includes: an executive summary; a report narrative of the findings based on cross tabulations selected by the consultant; frequencies, percentages, means and standard deviations of quantitative data; and content analysis of the textual data. The reports provide high-level summaries of the findings and identify themes found in the data. Generalizations for populations are limited to those groups or subgroups with response rates of at least 30%. The CSWG reviewed draft reports and provided feedback to the consultant prior to public release. The Campus Climate Survey results and reports were presented to the entire campus community in April 2017.
Ithaca College worked with the consultant to develop a research data security description and protocol, which includes specific information on data encryption, the handling of personally identifiable information, physical security, and a protocol for handling unlikely breaches of data security. The data from online participants was submitted to a secure server hosted by the consultant. The survey ran on a firewalled web server with forced 256-bit SSL security and is stored on a SQL database that can only be accessed locally. The server itself may only be accessed using encrypted SSH connections originating from the local network. Rankin & Associates Consulting project coordinator Susan Rankin has access to the raw data along with several Rankin & Associates data analysts. All Rankin & Associates analysts have CITI (Human Subjects) training and approval and have worked on similar projects for other institutions. The web server runs with the SE-Linux security extensions (that were developed by the NSA). The server is also in RAID to highly reduce the chance of any data loss due to hardware failure. The server performs a nightly security audit from data acquired via the system logs and notifies the administrators. The number of system administrators is limited and each will have had required background checks.
The consultant has conducted more than 170 institutional surveys and maintains an aggregate merged database. The data from the Ithaca College project was merged with all other existing climate data stored indefinitely on the consultant’s secure server. No institutional identifiers are included in the full merged data set held by the consultant. The raw unit-level data with institutional identifiers are kept on the server for six months and then destroyed. The paper and pencil surveys were returned to the consultant directly and kept in a locked file drawer in a locked office. The consultant destroyed the paper and pencil responses after they were merged with the online data. The consultant will notify the committee chairs of any breach or suspected breach of data security of the consultant’s server.
The consultant provided Ithaca College with a data file at the completion of the project.
The survey was administered to all faculty and staff at Ithaca College. Climate exists in micro-climates, so creating opportunities to maximize participation was important as well as maximizing opportunities to reach minority populations. Along these lines, the consultant recommended not using random sampling as we may have "missed" particular populations where numbers are very small (e.g. Native American faculty). Since one goal of the project was inclusiveness and allowing invisible "voices" to be heard, this sampling technique was not used. In addition, randomized stratified sampling was not used because we do not have population data on most identities. For example, Ithaca College collects population data on gender and race/ethnicity, but not on disability status or sexual orientation. So a sample approach could miss many groups.
This initiative included four primary phases. The first involved conducting focus groups (spring 2016), survey development (spring/summer) 2016), survey implementation that sought input from all students, faculty, and staff (fall 2016), reporting of results (spring 2017).