Standard 2: Planning and Evaluation
Keene State has a history of large scale, long-term, inclusive planning efforts on issues that affect general College policy and action. Administrative turnover and faculty work-to-rule in the period following the College's 1990 accreditation review resulted in an uneven implementation of "Vision 2000" planning goals. President Stanley Yarosewick, who arrived in 1994, sought new planning methods to re-engage the campus. In 1995 twenty-five members of the administration, faculty and staff participated in roundtable discussions of trends and forces in higher education. The roundtables were undertaken in conjunction with the Pew Higher Education Roundtable, a national research program sponsored by the Pew Charitable Trusts and based at the University of Pennsylvania's Institute for Research on Higher Education. The roundtable discussions revealed the campus's mistrust of "top-down" planning activities.
The process of building a sense of community and trust began with "Speak Out!," a three-day open space planning session held in May 1996 using large-scale system change techniques. Three hundred fifty participants came to identify issues that the campus community agreed were important to address. The proceedings from "Speak Out!" will be available in the workroom.
A second planning session, "Future Search," was convened in January 1997 with eighty participants, including members of the faculty, staff and administration, the Keene business community, the mayor, trustees, legislators, alumni, and regional K-12 educators and administrators. "Future Search" identified twelve themes which participants agreed must be addressed if the College is to achieve its desired future. Refinements of the twelve planning themes resulted in an institutional planning document, "Our Plan," disseminated in March 1998, which can be viewed as a crystallization of those ideas which gathered the greatest breadth of support during the two planning conferences. Since the creation of "Our Plan," decisions about needed planning efforts derive from it, and spontaneous ideas are commonly referenced to it. For example, directors submit to the appropriate vice-president an annual report on activities in their areas of responsibility, referencing relevant sections of "Our Plan."
In addition to the long-term planning effort represented by "Our Plan," more specific short-term planning activities have focused on facilities, enrollment management, academic programs, technology, faculty relations, and student services. The College-wide Planning Committee, which oversaw planning efforts from 1987 to 1992, subdivided into a group focusing on facilities planning and one focusing on enrollment management. In 1991 the College hired Sasaki Associates to work with us in the creation of a facilities master plan, approved by the Board of Trustees in January 1993, which has served as the basis for significant renovation, construction and fund-raising. The Facilities Planning Committee handles planning of specific project priorities and review of facilities plans and maintenance requests (see Standard Eight).
In January 1997 President Yarosewick convened a team of faculty and staff to work together to develop comprehensive recommendations for improving first-year retention. One recommendation resulted in the creation of the First-Year Council, co-chaired by representatives of the Student Affairs and Academic Affairs divisions, to coordinate a comprehensive program for first-year students. After evaluating existing programs, the Council focused on three initiatives: Academic Showcase, Orientation II, and First Semester Advising. The Academic Showcase, held on a fall weekend in the Student Center, displays student and faculty projects and aspects of the curriculum for our first-year students, parents, and prospective applicants. The First Semester Advisor program connects faculty and staff mentors with first-year students during their first semester on campus. During Orientation II, first semester students receive guidance regarding academic expectations, understanding a course syllabus, choosing an academic major and adjusting to college life. Each of these initiatives has been successfully implemented and leadership transferred to an existing College office. The First Year Council is continuing its planning and coordinating efforts (see Standard Four).
Because it became clear that many institutional issues crossed existing organizational boundaries, planning began about five years ago to integrate some functions of Academic Advising and Career Services into a new entity known as the Elliot Center. This union involved extensive work to integrate different office cultures, including developing a common physical space, a common budget and a single leadership. In 1999-2000 the Registrar's Office joined the Elliot Center to streamline information flow and organizational processes, in particular those supported by our student database, Datatel. Student Financial Services, the bursar, and the registrar (offices that had formerly reported to three different vice presidents) were initially united to assist in creating data structures that served the needs of each office and articulated with the data of the others. As the implementation has progressed and systems have become more fully operational, these offices have increased their collaboration with the Elliot Center. Planning continues toward the implementation of a comprehensive student services model.
To support retention initiatives such as changing our advisement services and creating the Elliot Center, Keene State and Plymouth State College recently received a grant from the New Hampshire Higher Education Assistance Fund (NHHEAF) to survey other campuses in New Hampshire regarding their first-year programming and the means used to evaluate its effectiveness. The grant project, which is just getting started, proposes the development and implementation of a model for evaluating first-year program effectiveness, and the dissemination of the result to other New Hampshire institutions.
The last decade has seen extensive long-term planning for and implementation of technology to support administrative and academic functions, beginning with a decision to wire the campus for voice mail and computer network access. A broadly representative technology committee was established to oversee the implementation of a technology plan, initially to ensure that all faculty members had computers on their desks, then that all students and faculty had e-mail and Internet access, and most recently to provide cable access throughout the campus, including residence halls. To support these efforts, the College developed a three-pronged support structure for end-users: Student Technology Support Services (STSS) to support student users, Administrative Information Systems (AIS) to support users of administrative systems, and the Center for Media and Instructional Technology (CMIT) to serve faculty in their teaching. We have created a help desk for technology needs, campus technology policies (Campus Network Utilization Policy - CNUP), decentralized computer laboratories, repair and replacement services, and a system for scheduled upgrades. These initiatives are described in greater detail in Standard Seven.
In the mid-nineties the College sought a relational database for student records to replace the existing limited (POISE) system. After a careful and extensive process, Datatel was selected as the vendor. The complexity of transition from one database to another has promoted a whole new set of planning processes. Resources from several administrative departments were combined to create a new office and staff, Administrative Information Systems, to oversee planning and implementation of information systems-most notably, but not exclusively Datatel. The need to adapt Datatel to our systems and vice versa has prompted extensive analysis of what data we need, in what forms, and the planning of both information and organizational systems to achieve that. In 1999-2000 President Yarosewick created the Student Information Systems Leadership Team to guide planning and implementation of Datatel functionalities.
The College has also adopted software for planning and implementing facilities maintenance and for scheduling the use of all campus facilities. The University System has purchased and begun planning for the implementation of a new financial, purchasing, human resources and research (grants) database (SCT Banner). Institutional and office needs were carefully evaluated in the creation of specifications for the bids. Like the implementation of the Datatel system at KSC, the University System-wide implementation of this new database will require extensive rethinking of functions, systems, and processes.
Intense competition for limited resources necessitates careful planning of fiscal priorities. For the most part such planning and management has been in the hands of the principal administrators. In 1998 the president reconfigured the budget advisory council to ensure that budget decisions were linked with campus priorities.
Planning for new degrees, majors and minors usually begins with departments or groups who develop a proposal, discuss it with divisional deans and with the Associate Vice President for Academic Affairs, then seek approval by the College Senate, the president, and the Board of Trustees Committee on Programs and Services. Decisions on adding or closing programs are generally made on campus, with the trustees and University System playing a role in final approval. There will be an increased emphasis on system-wide strategic academic planning among USNH institutions in the future. During the last ten years a major in Communication and a minor in Women's Studies have been added in response to student demand. The elementary, special education, and secondary education curriculum has been extensively revised, prompted in part by a new state requirement that students seeking elementary certification complete a major in an academic field. A new minor in International Studies was established, and Graphic Design was moved from Technology, Design, and Safety to the Art Department. We have streamlined associate degree programs and eliminated the Master of Arts in Liberal Studies and Master of Arts in Teaching.
Faculty planning typically occurs on three levels: staffing patterns (largely guided by student demand and funding levels), program development (normally initiated by individual or departmental initiative), and professional development. An additional planning and evaluation effort has emerged from changes in the mode of contract negotiation between the administration and the faculty union (KSCEA). After about fifteen years of relatively difficult labor history, contract negotiations shifted to a mutual gains model. Essential elements of this model have been the placement of some processes outside the formal structures of contract negotiation, and the development of a set of common data from which to work. Mutual study committees have been established dealing with workload, academic leadership (roles of chairs/coordinators), academic advising, health benefits, comparator institutions, and evaluation. The need for shared information has resulted in regular collection and dissemination of salary data and annual meetings to review data on workload and other issues.
Particularly useful has been the creation of a list of thirty comparator institutions (available in the workroom). While these have been used only modestly in contract negotiations, they have increasingly served as models and comparisons in a very wide variety of planning activities. In addition to the negotiation-produced comparators, the College's membership in COPLAC (Council of Public Liberal Arts Colleges) has provided an additional reference group for planning purposes (list available in the workroom).
The Office of Institutional Research was established in 1988 to create a centralized source of data, to prepare information for decision-making and strategic planning, and to establish on-going assessment and program evaluation. In addition to coordinating and providing data for decision-making and strategic planning, Institutional Research also completes external surveys and government reports and is the clearinghouse and consultant for internal survey research. Over the past eleven years, the office has created a student tracking system from which retention analysis, graduation rates, and student success segmentation analysis are performed. The First-Year Council and the Administrative Assessment Committee have used results from these analyses in the decision-making process. A faculty workload database has been created yearly since 1994. A standard database is provided for those programs under review by the Academic Overview Committee, including data on enrollments, advising loads, and faculty workloads. Extensive data is provided to deans and department chairs on course enrollments and numbers of majors and minors to assist in planning courses and numbers of seats needed to meet student demand. Much of this data appears in the annual Factbook which is now available in electronic form as a planning resource on the College's network.
In 1993 the College began evaluating our continued participation in NCAA Division II sports competition. After careful evaluation of the respective costs and benefits of Division II and Division III competition (earmarked scholarship dollars, travel costs, administrative costs, levels of student participation) a decision was made to begin a four-year transition to Division III, which was successfully completed in 1998.
Over the last ten years the College has regularly utilized outside consultants in support of all major planning efforts: the development of "Our Plan," technology planning, facilities planning, enrollment management and the integration of the formerly separate components of the Elliot Center. Consultants have provided an important source of external expertise and perspective without relinquishing local authority over the final form of proposals, decisions and their implementation.
Short-term planning occurs in a number of administrative bodies such as the Deans Council. Other committees that contribute to long- and short-term planning are the Facilities Planning Advisory Council, the College Budget Council, the College Assessment Committee, and the College Information Technology Committee. The College also establishes task forces to deal with particular planning issues. Examples include the General Education Task Force, the First Year Council, and the Calendar Committee. Most such groups include all affected constituencies: faculty, professional administrative and technical workers, operating staff, and students.
Prompted by the University System Board of Trustees, assessment strategies and programs have been emphasized since the late eighties. In 1988, when the Office of Institutional Research was created, an initial survey of "student outcomes" data found that large amounts of data were collected, poorly organized and poorly communicated, and that virtually none of it directly evaluated student learning. As a part of this student learning outcomes audit, a number of data collections began including a syllabus survey, transcript survey, senior survey, and alumni survey, and have been repeated at regular intervals since. In 1998 a second assessment survey was conducted, an overview of all data collections and reports generated on a regular basis and utilized in policy development and decision-making. This document will be found in the workroom. Following the student learning outcomes audit in 1988 we received an Innovation and Opportunity Grant to organize assessment databases and develop a plan for student outcomes assessment. The data collected and those databases have become a significant part of the continuing work of the Office of Institutional Research. A list of reports and databases created by this Office will be found in the workroom. An Administrative Assessment Committee was established to identify data needs and to coordinate collection and distribution of those data. This committee has played a significant role in streamlining subsequent data collections and analysis used for enrollment management.
In order to further faculty acceptance of assessment, the Vice President for Academic Affairs created a Faculty Assessment Committee in 1998. This group worked to create a "culture of evidence" through initiating collection, evaluation and dissemination of data on reasons students take longer than four years to graduate, attitudes toward the add/drop process, and the content of course syllabi. The group also conducted sessions during PEPT week (a faculty development program held annually at the end of spring semester) to educate faculty about student outcomes assessment. It worked with the Vice President for Academic Affairs to promote faculty readiness to develop and implement more effective academic assessment. The work of the Administrative Assessment Committee and the Faculty Assessment Committee has been taken over by the newly created College Assessment Committee.
Currently, the Director of Institutional Research supports the campus evaluation process by designing models for continual assessment of programs and student progress, such as satisfaction surveys, and senior and alumni surveys, and by furnishing data for program review of academic departments. Students' success is monitored by tracking them from their admission and freshmen CIRP data, through to their alumni survey results. Segmentation analysis is performed on the retention data from the student tracking system to determine which students are successful and which are having difficulty, dropping out, or transferring. The alumni survey has been implemented since 1993 for one- and five-year out alumni classes. The results of the alumni and senior surveys provide information for the indirect assessment of student outcomes.
Evaluation efforts have steadily increased in number and sophistication. Data collection has become an expected accompaniment of all proposals for change. For example, in recent years proposals have been made for changes in the calendar, in time blocks for classes, in the add/drop policy, the advising system, student course evaluation form(s), the general education curriculum, the move from Division II to Division III, and the creation of a Communication major. All collected extensive data preceding final decisions. Despite the prevalence of data resistant opinions on some college campuses, it has become standard practice at Keene State to seek data from all accessible stakeholders prior to making decisions.
Smaller scale assessment planning has also taken place in the First Year Experience program (now replaced by the First Year Council and First Semester Advisors), the FIPSE-funded Multiple Perspectives Teaching project, and work on the new teacher education curriculum.
Since 1998 all self-studies prepared by departments for their program review include assessment plans. These regular evaluations of academic programs take place under the auspices of the Academic Overview Committee (AOC) of the College Senate, which is comprised of six senators (one a student), three faculty (one elected from each division), one student elected by the Student Assembly, and a member of the campus community designated by the Vice President for Academic Affairs. Programs reviewed every seven years include not only academic departments but also offices that support the academic mission of the College: the Academic and Career Advising portion of the Elliot Center, National and International Exchange, Continuing Education, etc. Programs conduct a self-study according to guidelines developed by the AOC and utilize standard databases available from the Office of Institutional Research. A sub-committee of the AOC works directly with each program, assists in the selection of external reviewers and the coordination of their visits, receives their report and writes a final summary report for the Senate. Initial resistance has not been eliminated, but increasingly, the value of periodic reflection and evaluation has come to be appreciated. These reviews often serve as the basis for goal reviews, increases in staffing and curriculum changes, as well as providing information useful in arguing for additional budgetary support. Programs which are accredited by professional associations, state, or national accreditation bodies utilize that self-study as the foundation for their AOC program review, thus reducing duplication of effort.
Review of co-curricular programs has followed a similar but less systematic process. The AOC has established a schedule to review co-curricular programs that support the academic program. While programs such as the Elliot Center and the Writing and Math Centers are slated for review according to a schedule printed in the Faculty Manual, neither procedure nor timeline exists for evaluating the effectiveness of offices not related to the curriculum.
All employees are regularly evaluated by themselves and their superiors. These evaluations are formal, written, and focus on providing information useful in improving performance. Personnel complete annual reports of their activities and projections of their goals. Faculty evaluation is formally defined by Article VII of the KSCEA Collective Bargaining Agreement; additional guidelines mutually agreed upon by the KSCEA and administration can be found in the Faculty Manual. As a consequence of the recently negotiated 1999-2002 agreement, department chairs have replaced discipline coordinators (the terms in use for the past twenty years). Chairs have a more explicit set of expectations, although evaluation of faculty is not among them. Over the years there have been several discussions of the need for faculty to evaluate divisional deans; to date no such process has been implemented, although deans are evaluated by the Vice President for Academic Affairs. Evaluation of principal administrators, professional and technical staff, and operating staff follows System personnel procedures.
Our extensive planning has proven effective in preparation for implementation where the planning is narrowly focused and occurs within existing structures. Some specific and local initiatives have supported institutional movement. Discussions of general education, combined with program review, have prompted several departments to rethink their curricular goals and structure. Discussions of assessment have prompted thoughtful consideration and some departmental initiatives, particularly in Education. The Campus Master Plan has become the driving force for the renovation and improvement of the campus-with great success. Where planning has been broader, the tendency to look for consensus has often impeded implementation. With the development of "Our Plan" all constituencies had the opportunity to participate. While the "something-for-everyone" process was necessary for campus buy-in of the planning effort, the result caused some sacrifices in clarity and in utilizing the plan as a reference system for setting priorities and making decisions. Overall, however, "Our Plan" has generated broad-based consensus and has become a basis for working toward institutional goals.
Planning has been hindered by budget uncertainties and the shifting patterns of student interest. Planning the number of seats to have available, and in which classes, has been quite uneven and only quite recently shown improvement. One of the factors that has limited the use of good data in planning has been the lack of a readily available data warehouse.
Although we are still in the beginning stages of developing campus-wide consensus on the need for assessment, we have taken some essential first steps. The Institutional Research Office now has a lengthy list of regular reports that are widely distributed. Program review has become accepted and, in fact, valued by many. All program review self-studies must address program assessment. However, except for those areas that undergo program review, subsequent evaluation usually occurs only when changes are proposed. Furthermore, we need to clarify the role of the College Assessment Committee and its relationship to other governance structures.
Beyond completion of requisite courses and student grades, assessment of student learning and the identification of explicit learning goals remain uneven. New curriculum proposals must include statements about future assessments of student learning. Some programs, most notably Education, Special Education and Early Childhood (ESEC), are implementing learning outcomes assessment within their major. Portfolios are used in Education, Art and Graphic Design, recitals in Music, a senior project in Communication and a research project in Economics. Several programs (Computer Science, Management, Psychology, Safety, Women's Studies, Theatre and Dance) have capstone courses. Some programs, especially in math and science, are sequential: students must take courses in a prescribed order and must have successfully learned the material in prior courses if they are to succeed in those that follow. Some programs conduct employer satisfaction surveys. No programs conduct independent external assessments of student work or have explicit grading standards for student work.
Over the next ten years we will participate in planning efforts at the System level, where the new chancellor, Dr. Stephen Reno, has expressed interest in coordinating planning activities across individual campuses. The NEASC re-accreditation self-study process has stimulated the examination of institutional issues and will be instrumental in the next stages of our planning process. We will emphasize the integration of our data collection into "Our Plan" decision-making, and clarify our procedures for controlling access to data. In areas where planning and implementation have not been well linked, we will work to clarify the appropriate governing authority and better articulate those planning processes with decisions and implementation. We will establish specific regular data collections and processes for their distribution to produce optimal influence on planning. We will also establish a clearer process for the management and distribution of occasional data requests. We will continue our use of large-scale, participatory planning processes, both general and more specific, with more emphasis on collaborative efforts across traditional organizational boundaries such as the Elliot Center. Adoption of the Datatel student information software and SCT/Banner financial and human resources software will continue to promote more collaborative and integrated efforts, and facilitate access to appropriate and usable data for more effective budget and resources allocation planning. With the increased availability of data an important priority is the clarification of who will have access to what data for planning purposes. As an example, a plan is being developed to identify the specific data needs of deans and department chairs for their planning purposes and to provide those data in the formats which will best serve those needs. Efforts will continue to establish clear and formal goals for general education and for other programs that can be used as a basis for planning as well as evaluation.
The College Assessment Committee will oversee and coordinate evaluation activities. The committee will not actually collect data or be responsible for sharing it, but rather will ensure that appropriate systems are in place, evaluate evaluation efforts, set policy, and recommend new directions. Evaluation models created in ESEC, English, and Biology will be used in developing academic goals and evaluation plans in other academic areas. Campus awareness of expectations for goals and assessment plans within the Academic Overview process and in the review of new curricular proposals will increase as a result. The committee will work to support these developments and promote similar endeavors in other areas of the College. Under the direction of the committee work will continue to streamline data collection and distribution. While we will undoubtedly increase the use of data as a basis for planning, we must also refine our procedures for determining who has access to secure data.
The College will re-examine the Senate's role in the program review process including the charge to the Academic Overview Committee concerning institutional assessment and the question of co-curricular program review. Lead examples in the development of effective assessment will continue to be the work of the First Year Council and the Elliot Center. We will continue to emphasize functional collaboration and the evaluation of functions, not just focus on specific organizational units.