Assessment Plan

Purpose and Vision

We believe that a robust, independent media is fundamental to a democratic society in which individuals are empowered as critical thinkers, creative problem-solvers and engaged citizens. We expect our students, faculty and staff to use their communications skills and expertise to help our communities adapt and thrive in a complex, global society.

We aspire to be a catalyst for positive change in our region and a leader in modern media education.

---

Curricular Assessment Goal Statement
Bachelor of Science in Journalism (BSJ)

The WVU Reed School of Media and Communications prepares its students to excel as professional communicators, scholars and innovators in a rapidly changing global media environment. As such, the School maintains the following overarching learning goals for its Bachelor of Science (Journalism and Advertising and Public Relations) students.

Upon completion of the BSJ, students will:

• demonstrate professional communications knowledge, skills and judgment
• demonstrate the ability to work professionally and effectively as part of a diverse team
• understand and apply the U.S. principles and laws of freedom of speech and press, understanding them within a broader global context
• understand the range of systems of freedom of expression around the world, including the right to dissent, to monitor and criticize power, and to assemble and petition for redress of grievances
• demonstrate an understanding of the multicultural history and role of professionals and institutions in shaping communications
• demonstrate culturally proficient communication that empowers those traditionally disenfranchised in society, especially as grounded in gender, race, ethnicity, sexual orientation and ability, domestically and globally, across communication and media contexts
• understand concepts and apply theories in the use and presentation of images and information
• demonstrate an understanding of professional ethical principles and work ethically in pursuit of truth, accuracy, fairness and diversity
• think critically, creatively and independently
• conduct research and evaluate information by methods appropriate to the communications professions in which they work
• write correctly and clearly in forms and styles appropriate for the communications professions, audiences and purposes they serve
• critically evaluate their own work and that of others for accuracy and fairness, clarity, appropriate style and grammatical correctness
• effectively and correctly apply basic numerical and statistical concepts
• apply tools and technologies appropriate for the communications professions in which they work

Direct Measures:

• The Theory, History, Ethics, Law, Digital and Diversity (THELDD) Test is given to JRL 191 and to capstone students at least once every two years. Goal is for students to earn an average of at least 70% on the exam. (latest exam: Spring 2021)
• Program capstone evaluations take place at least once every two years. (latest evaluations: Spring 2021)
• Internship supervisor evaluations are ongoing and compiled and assessed at least once every three years.

Indirect Measures:

• A syllabus audit is conducted at least once every three years to ensure competencies remain across program areas. (latest audit: Fall 2021)
• Graduating senior exit survey is conducted each December and May.
• Recent graduate (previous three years) alumni survey is conducted every three years. (latest survey: Summer 2022)

This assessment plan will be revisited and updated at least once every three years (evaluation measures were updated in academic year 2019–2020). The THELD test was revised in 2018 to become the THELDD exam and include questions specifically related to digital competencies; three additional questions related to diversity (2) and theory (1) were piloted in Fall 2021.

Results of assessment measures and responses are included in separate reports, which are shared with faculty as a whole and/or as part of their respective curricular programs (i.e. Journalism, Advertising and Public Relations).

---

Theory, History, Law, Ethics, Diversity and Digital (THELDD) Test Background

An exam (the annotated version of which follows this document) to measure all graduating students’ knowledge of key history, law and ethics concepts was developed by faculty and initially tested in Fall 2007, when it was administered to all Journalism 101 students on the first or second day of class and to all Journalism seniors via their capstone courses at the end of the term.

The HEL test was subsequently further honed and refined twice via faculty review, with confusing and more trivial questions deleted to get the test down to 50 questions (from 60 in the first administration in 2007, to 57 questions during its administration in Fall semester 2008 and Spring semester 2009). In 2014, it was pared down to 35 questions, which also included questions related to diversity and communication theory. (Questions that were deemed too major-specific, e.g. public relations-based, or that were considered more superfluous or poorly constructed were deleted, with new questions added from instruments successfully used by our colleagues at Elon University and the University of South Carolina.) In 2018, questions relevant to digital technologies knowledge were added; in 2021 and 2022 additional questions were added around theory, diversity and intercultural competency. This exam has been given every two years to our freshman cohort in their introduction to media and society course (JRL 101/MDIA 101) and our graduating senior cohort through their respective capstone courses (e.g., JRL 459, ADPR 459).

Because of poor scores over time (< desired 70% correct), the school's assessment committee, led by the curricular program chairs, were tasked with recommending a more aggressive approach to ensuring these key concepts are known by graduating students. Past approaches have focused on distributing the annotated exam and results to faculty and encouraging them to reiterate the key concepts in applicable courses. However, this approach has not resulted in the desired outcome, so a more aggressive approach was deemed necessary. Program chairs met with their respective faculty and identified specific courses where these concepts could be reiterated. As a result, the last graduating cohort (AY 20-21) assessed scored the highest mean yet (68.42% correct), although still under the desired mean of 70% correct.

The THELDD exam continues to be administered to incoming and graduating cohorts at least once every two years. These scores are analyzed for statistical differences between the groups and maintained to monitor these cohorts upon beginning and ending their studies.

---

Advertising and Public Relations Major

The Reed School of Media and Communications states as its learning goals the values and competencies of its national accrediting body, the Accrediting Council for Education in Journalism and Mass Communications, which appear under information about the B.S. in Journalism degree. In addition, the School faculty have set other specific educational outcomes deemed critical for success as professional communicators. These additional educational outcomes for advertising and public relations majors are:

• Advertising and public relations graduates will understand how to serve, reflect and engage diverse publics and will be prepared to either work in the field or to pursue advanced educational opportunities.
• Advertising and public relations graduates will demonstrate professional competency in preparing campaign plans, using both traditional and digital means, including obtaining, analyzing and interpreting data; establishing goals and objectives; identifying appropriate strategies; developing creative tactics; and understanding budgeting, timeframes, and success indicators/evaluation.
• Advertising and public relations graduates will demonstrate an understanding of the history of media, advertising, public relations, and the influence of technology on the communication professions.
• Advertising and public relations graduates will demonstrate the ability to professionally present ideas in all forms: written, verbal, and with the use of appropriate digital/electronic audio-visual materials.
• Advertising and public relations graduates will understand the working relationship between advertising and public relations, as well as related marketing communications vehicles and media planning and placement.
• Advertising and public relations graduates will be able to demonstrate knowledge and understanding of communication ethics and law as it applies to advertising, media and public relations, including privacy in the context of database marketing, artificial intelligence and social media.
• Advertising and public relations graduates will be able to work effectively and collaboratively in teams to create messages, solve problems and develop and implement integrated communication strategies using human-centered design principles.

Assessment Measures & Standards

Advertising and Public Relations graduating seniors will be required to produce a campaign and proposed or actual tactics in the capstone course that includes research, objectives, strategy, media plan, creative execution and evaluation techniques. Alumni and/or local professionals, along with at least one non-capstone affiliated ADPR faculty member evaluate approximately 25 percent of the portfolios, chosen at random, at least once every two years to measure major learning objectives and ACEJMC competencies and professional values. Means across the reviewed campaign plans should measure at least at 3.0 on all evaluated dimensions on a five-point Likert scale.

• The School of Media and Communications conducts a Senior Exit Survey annually. This survey measures students’ satisfaction with the program, including quality of education and their perception of their skills/competencies learned.
• The School of Media and Communications conducts an Alumni Survey of the prior three graduating classes at least once every three years. This survey measures whether the journalism program provided graduates with the education and skills needed to gain and maintain professional employment in their discipline or allied fields.
• In the capstone course, students are required to work in teams, creating campaigns for actual or hypothetical clients in agencies, companies, and/or nonprofit organizations. Students’ individual performances within their teams in the required ADPR 421 and/or ADPR 459/ADPR 457 courses are evaluated by peers. The evaluation measures the contribution of individual teammates’ work. Professors adapt team assignments and evaluation instruments in these courses as needed, based on these results, and share team assessment tools with each other, as needed/desired. (Team evaluation forms are available for review.)

Assessment Tool

In the 2019-2020 academic year, the assessment tool was updated twice. The first update was to better reflect our updated program-level educational outcomes. Specific revisions focused on ways to better assess diversity and digital related learning. The new assessment tool was piloted and after reflecting on the reviewer feedback, wording was further refined in the spring. Additionally, new response options were added (NA, CD, NP) so that it might be more clear why low scores might be occurring in key areas. The revised assessment tool was again tested at the end of spring 2020.

AD/PR Capstone Assessment Measures

Students use a 1-5 likert scale with additional options for Not Applicable to Campaign (NA), Couldn’t Determine (CD) and Not Present (NP) to respond to the following statements regarding the capstone campaign project.

• Research (secondary and/or primary) is Appropriate.
• Research is presented well visually/graphically.
• Target audiences are identified and appropriate with diverse population considered.
• Campaign plan includes an appropriate SWOT analysis or Problems & Opportunities.
• Target audiences are identified and appropriate with diverse population considered.
• Campaign plan follows an organized planning process.
• Positioning statement is clear and appropriate for the identified audiences.
• Consistent key strategic messages that are appropriate for the identified target audience(s) are provided.
• Strategies and tactics are appropriate and realistic for the available budget
• Campaign plan is realistic (i.e., recommends appropriate timeline, budget, media, deliverables).
• Creative materials are appropriate for the identified target audience(s) and available budget and reflect the campaign concept.
• Objectives are specific, measurable, realistic, deadline-oriented and appropriate to the campaign with the given timeframe.
• Media plan is appropriate for the identified target audience(s) and for the available budget.
• Digital media and/or communication technology is integrated into campaign as relevant.
• An appropriate, realistic evaluation plan is employed in the campaign plan.
• Campaign plan is well written, concise, accurate and grammatically correct.
• The campaign, target audience(s), and/or execution consider culturally diverse perspectives and inclusivity (e.g., socio-economic status, race/ethnicity/nationality, marginalized populations/communities)
• Campaign plan is well organized/presented (logical, professional and stylistically consistent).
• Campaign plan demonstrates an appropriate use of visual/graphics/design.
• Campaign plan book is professionally presented.

Qualitative Information

Students provide written comments/suggestions about (a) any especially strong areas of the project and (b) how the project/course can be improved. Results are provided in the aggregate to faculty upon request.

• Notable Positives (Specific to individual campaign plans.)
• Room for improvement (project and/or course)

Use of Results for Program Improvement
Results are shared with faculty and decisions made as to how to address academic/competency deficiencies in the coming year.

---

Journalism Majors

Expected Educational Outcomes

The Reed School of Media and Communications states as its learning goals the values and competencies of its national accrediting body, the Accrediting Council for Education in Journalism and Mass Communications, which appear under information about the B.S. in Journalism degree. In addition, the College faculty have set other specific educational outcomes deemed critical for success as professional communicators. These additional educational outcomes for journalism majors are:

• Journalism graduates will demonstrate proficiency in critical thinking skills, writing and reporting, and an understanding of basic production skills, allowing them to produce news stories and multimedia projects. Graduates will be adequately prepared to either work in the field or pursue advanced educational opportunities.
• Journalism graduates will demonstrate a mastery of written and spoken communications, an understanding of the technologies of print, television and digital media, and knowledge and applications of these skills in their chosen careers.
• Journalism graduates will demonstrate an understanding of how to serve, reflect and engage diverse publics in their reporting and producing.
• Journalism graduates will demonstrate knowledge of media ethics, law and regulation, including a full understanding of the First Amendment within the context of artificial intelligence and social media.
• Journalism graduates will demonstrate specialized knowledge of news media interactions with various critical publics, including but not limited to: government at all levels; educational entities; law enforcement; medical, social and humanitarian services; and religious and secular organizations within the community.
• Journalism graduates will learn to work as collaborative teams to solve problems, create strategies and produce content across all media platforms using the principles of human- centered design.
• Journalism graduates will demonstrate the ability to engage an audience using social media networking and analytics tools.
• Journalism graduates will demonstrate an understanding of the history of technology and Silicon Valley in the context of media and its impact on acquisition, production, distribution and the economic models of media.
• Journalism graduates will learn methods for data mining, evaluating sources, and investigating algorithms.
• Journalism graduates will be introduced to programming for media in one or more modern languages.

Assessment Measures and Standards

Journalism graduates are required to produce a minimum of two multimedia projects. Alumni and area professonals, along with at least one non-capstone faculty member evaluate the final multimedia projects at least once every two years. These projects demonstrate proficiency in writing, reporting and the ability to tell an effective story through multiple digital technologies, including images, audio, text and graphics.

• The School of Media and Communications conducts a Senior Exit Survey annually. This survey measures students’ satisfaction with the program, including quality of education and perceived learning of ACEJMC skills and competencies.
• The School of Media and Communications conducts an Alumni Survey of the past three year’s classes every three years. This survey measures whether the journalism program provided graduates with the education and skills needed to gain and maintain professional employment in their discipline or allied fields or to successfully pursue graduate work.
• Individual contributions to the student capstone teams are assessed by the professor as student work is iterated and evaluated.
• The highest quality stories may be picked up by regional media outlets, providing another measure of quality work.

Assessment Tool: Journalism Capstone Assessment

Courses being Evaluated: JRL 459 Multimedia Bureau Reporting, JRL 487 WVU News or JRL 431 Multimedia Storytelling

Background: In all capstone classes (reporting, visual and television focused), students work in teams to produce multimedia packages. Their projects can be a mix of text, photos and video, or the piece can also include text, interactive graphics, and audio slideshows. These packages are what our assessment team of professionals and academics evaluated. Results will be shared with faculty and decisions made as to how to address academic/competency deficiencies in the coming year.

The assessment team is asked to respond to the following statements regarding the students’ “pilot” converged capstone multimedia projects, using a 1-5 likert scale with an option for NA:Not Applicable.

• Story is well researched and reported thoroughly with the correct official/expert, statistics, includes both sides, and/or provides a different angle or perspective. (Research Report, Multimedia project)
• Demonstrates an understanding of how to critically evaluate individual and team work and use hyperlinks in producing a multimedia project. (Blog Postings)
• Demonstrates technical proficiency when selecting, reporting and producing broadcast, print, web and multimedia stories. (Multimedia Project)
• Multimedia project is newsworthy, relevant and visually interesting to the audience. (MM Project)
• Demonstrates a basic understanding of using video and still camera equipment. (Multimedia Project)
• Demonstrates the ability to select the correct medium (video, text, photos, graphics) to correspond with the story selection. (Multimedia Project)
• When appropriate, the story includes multiple viewpoints, and diverse perspectives, including those of women and minorities. (Race, gender, age, demographic and socio-economic class, background) (Multimedia Project, Research)
• Demonstrates the ability to tell an effective story through images, text and graphics. (MM Project)
• MM Project is technically proficient, easy to navigate, and includes different information across multiple platforms. (Multimedia Project)

The assessment team is also asked to provide qualitative comments.