The Graduate Center, City University of New York
The Performance Management Process (PMP) of The City University of New York (CUNY) uses metrics to measure standardized goals for its college’s use of technology, and insufficiently recognizes online and hybrid courses as experimental and experiential research and pedagogy. The metric used to evaluate online learning encourages colleges to use innovative technology to improve teaching, yet measures solely by the number of courses offered fully or partially online. PMP numbers are self-reported by college administrators and data indicate discrepancies in percentages of students enrolled in online or partially online courses and those reported on the PMP.
The proliferation of hybrid-online courses serves a dual purpose related to the standards outlined in the PMP. First, college executives, including presidents and vice-presidents of colleges, earn salary increases if targets are met. Second, colleges can improve other target areas that value for cost-effectiveness and efficiency. By offering more online or partially online courses, each institution can increase enrollment numbers by capitalizing on unused facility space, while concurrently increasing the number of matriculated students paying tuition. The recommendations provided in this brief illustrate a desire for the expansion of the PMP to include incentives for faculty interested in researching innovative interdisciplinary approaches in hybrid-online teaching, and for an investment by the university to support research in non-traditional virtual environments.
STATEMENT OF PROBLEM
Online and hybrid courses have enormous potential to braid marketplace demand for efficient higher education with innovative digital technology and cutting-edge pedagogy. While many institutions have adopted learning management systems as hubs for hosting online courses, some institutions have stretched the market by enhancing their digital delivery systems with electronic portfolios, video response programs, and gaming software. However, universities still grapple with the central purpose of promoting online learning. The City University of New York (CUNY) launched the Hybrid Initiative in 2009 as part of its university-wide technology initiative. Presently, the university is focused on achieving the metrics of the Performance Management Plan (PMP), which indicates that colleges are increasing online and hybrid course offerings (defined as courses where 33-67% of work is completed online). While campuses across the University are working to hybridize courses and develop best practices in online teaching and learning, administrative support from the central office has been historically unsteady. However, the PMP metric remains intact and campuses are held accountable for increasing the number of students enrolled in online or hybrid courses.
CUNYfirst, the University’s multi-million dollar operating system, indicates that campuses can choose from a variety of online course offerings, ranging from online to partially online, as well as hybrid and web-enhanced. Yet, the default for all courses is face-to-face, an issue which university officials hope to address. With a limited range of choices for departments to choose from when entering course schedules, and with little centralization regarding the proper terminology used to report the use of online courses, the scope of the use of online and hybrid courses across the University is currently unclear. CUNY’s Office of Institutional Research has remarkably little data on the number of students participating in online or hybrid courses. Hence, the relationship between the PMP goal of increasing online and hybrid courses institution-wide and whether the data being reported in the PMP are accurate and reflective of a common mission, must be better understood.
At present, there also appears to be an inverse relationship between the need for strict accountability and growth in the area of technology and pedagogy, and institutional support, including the development of a common understanding of terminology. The purpose of this brief is to encourage CUNY to align its demands for accountability and growth in online and hybrid learning with the proper institutional support university-wide.
CUNY has a robust history of scholarship in myriad areas of instructional technology. In 1993, the CUNY Office of Budget, Finance, and Computing underwent a full-scale restructuring to bring the frontier of information systems to the colleges in a meaningful way (Rothbard, 1994). Key stakeholders in this reorganization developed “the Open Systems Center, a high-end research and training facility located in the Computer Information Services office at the University’s central hub. The center was designed to serve as a test-bed for the application of new technology to problems encountered by the University’s professional staff in teaching, research, and administration” (Rothbard, 1994).
One of the primary goals of CUNY’s Director of Education, Training, and Staff Development, Colette Wagner, was to maximize the services of the physical space to support the research of innovative practices in teaching with rapidly developing technologies. A core group of four CUNY faculty initially brought together a team of visiting fellows, who focused on the Center’s main objective to “provide an experimental environment for prototyping instructional software that [could] be used in the real world of teaching, learning and research at CUNY” (Wagner, 1994). As the team explored the blurry intersections between teaching and technology as pioneers in the field, they also led a range of professional development workshops and seminars, some of which attracted over a thousand faculty members from across the CUNY system. The Open Systems Center puts faculty and staff training at the core of its mission. The team suggested that its “long-term goal [was] to use this information to enable the successful negotiation of a University-wide training contract that [would] again provide colleges with lowest possible costs for upgrading staff skills in this crucial area” (Wagner, 1994). In the mid-1990s, CUNY faculty and administrators were partners in cutting-edge research in pre-World Wide Web instructional technology. As the decade progressed, members of the Open Systems team were awarded grants that would shape the pedagogical landscape of the University.
In 1997, CUNY launched its first asynchronous course at Hunter College. Professor Anthony G. Picciano offered an online version of an education administration course, serving New Yorkers whose busy lives prevented them from enrolling in a traditional course and for whom the online environment could integrate more seamlessly into their schedules (Picciano, 1998). A $60,000 grant allowed this pilot to provide positive evidence for the use of online courses, prompting the Alfred P. Sloan Foundation to fund a $300,000 planning grant for a University-wide program that would offer faculty and staff training opportunities on the hardware and software needed to effectively build online courses (Otte, Pecorino, & Picciano, 2001). The team developed “more than 80 courses at a dozen campuses” including hybridized versions of fully online classes. The following year, the Sloan Foundation awarded the group, led by Picciano, $2 million to support a University-wide, three-year project (Otte, Pecorino, & Picciano, 2001). During this time period, many other faculty administered, or participated in, grant-funded research programs that generated a wealth of information, best practices, and pedagogical possibilities for teaching and learning in online environments. Many of these research collectives have since dissolved. When Matthew Goldstein was appointed Chancellor of CUNY in 2000, research in instructional technology and online learning was superseded by a push to implement broad changes in institutional accountability and to efficiently increase graduation rates. In turn, even as universities broadened the use of instructional technology, a new set of problems arose in their attempt to balance efficiency and meaningful pedagogy. Institutional policies and accountability measures have led to a repurposing of online and hybrid learning for ease of accountability and for the mass-market production of courses.
The Performance Management Process and Discrepant Data
At the start of Goldstein’s tenure as Chancellor in 2000, the twenty-three colleges of the CUNY system were notified that strict management accountability measures would be instituted as part of an annual review process (McDonnell, 2013, p. 4). The Performance Management Process (PMP) was designed to unify the network of CUNY colleges around specific targets and measureable goals. The second, and most influential goal of the PMP initiative, was to hold each campus accountable to key performance drivers attached to specific metrics (McDonnell, 2013, p. 4). Each year, college presidents work with CUNY vice-chancellors and university deans to establish measurable benchmarks that serve as a snapshot of progress or failure in key areas of policy, structure, teaching, and efficiency (ibid). Not only do these metrics illustrate the capacity of the college to perform in relation to other CUNY institutions, success in each area determines the amount of each college president’s salary increase the following year (ibid). Consequently, if achievement is identified in all measurable areas, members of the college’s executive staff (vice-presidents, deans, etc.) are also eligible for additional salary increases.
The PMP currently prioritizes institutional achievement in three areas. The first measureable outcome is to “raise academic quality” (McDonnell, 2013, p. 4). Secondly, the PMP evaluates colleges on their ability to “improve student success” and to enhance “financial and management effectiveness” (ibid). Each of these broad areas is sub-divided into an array of specific expectations that are each measured using a quantitative measurement system. Moreover, “metrics are carefully designed to incentivize desired behavior by the college” (ibid). In order to demonstrate compliance with the standardized behaviors, college presidents submit an annual report to the CUNY Chancellor for review, discussion, and feedback.
Under Goal One of the PMP, the university notes that its first objective for colleges is
to “strengthen college priority programs and continuously update curricula and program mix” (The City University Performance Goals and Targets 2013-2014, 2013, p. 1). One of the key outcomes of this objective is to ensure that colleges use technology to “enrich courses and improve teaching” (ibid). The quantitative measurement for this objective is determined by the percentage of instructional Full-Time Equivalents (FTE) offered fully or partially online (ibid). Each college reports a numeric figure to the Chancellor using an algorithm that calculates the number of students enrolled in full or partially online courses.
PMP data present a range of discrepancies leading to further questions about the reliability of reported information. These data are self-reported to the University’s Central Office by the administration of each campus. While the metrics that determine how the data are represented in the final version of the PMP are standardized, information gathered by individual campuses illustrate that compliance is widely varied from institution to institution. A senior University official noted that the central registration and reporting system, CUNYfirst, is ineffectively set up to aggregate university-wide data in a number of key areas. In fact, CUNYfirst has made it impossible for the University to gather data on exactly how many online or hybrid classes are offered and how many students are enrolled in such courses. The University’s Office of Institutional Research has no reported data on online learning. Within CUNYfirst, online courses have a range of complex and nuanced definitions. Campus departments and registrars can approach each of these variations differently, leading to inaccurate cross-campus reporting of the number of students enrolled in online or hybrid courses. The Borough of Manhattan Community College eLearning site lists the following types of online courses:
1. Fully Online: There are no face-to-face meetings during the semester.
2. Hybrid: 33%-80% of the course meets online. The remainder meets in person.
3. Online: The majority of the course meets online, but students may be asked to meet in the classroom on a number of occasions.
4. Web-Enhanced: The course meets in the classroom, but it relies heavily on an online component to enhance instruction.
These taxonomical distinctions have generated inaccurate and unusable PMP data. For instance, the 2014 PMP report summarizes University Target 1.3: Colleges will use technology to enrich courses and improve teaching that is evaluated by the percentage of Instructional (student) FTEs offered partially or totally online for the previous five years of data in this area. The web-enhanced category provides a catchall for courses that administrators can argue use the Internet, including the Learning Management System, Blackboard, for instruction. However, there is no shared understanding of what is considered web-enhanced and how face-to-face courses differ in classification. These shades of grey call into question the validity of the reported data presented in the annual PMP and whether this form of institutional accountability is effective.
In the Fall of 2011, Kingsborough Community College reported that 18.3% of its FTEs were enrolled in partially or totally online courses (CUNY Office of Institutional Research, 2014, p. 2). However, the following year, Kingsborough’s number dropped dramatically to only 3.2%. In the same year, Kingsborough’s FTE only dropped by 644 students, which would not explain the radical decline in those enrolled in online or partially online courses (“Five Year Trends in Enrollment,” 2013, p. 1). With little cohesive data and system-wide inattention to accurate reporting, the University’s stated aims of accountability seem, at present, much more forceful than its actual commitment to raising academic quality in this area. Transparent and accurate reporting of online and hybrid course offerings and enrollment data will guide the University in moving fiscal resources toward colleges and programs which are committed to enriching digital courses. Without a clear picture of the institution-wide landscape there is limited possibility for strengthening programs already underway or allocating resources to develop new initiatives.
Narrow Institutional Focus
A recent push to implement a broad-spectrum reform in the way the university conducted online and hybrid learning was evident in the 2009 creation of the Hybrid Initiative. The implementation of the first working group and the call for proposals to institute online and hybrid courses across the college system indicated that CUNY administrators were placing increased value on the instructional possibilities of technology-rich environments. While the Hybrid Initiative was a successful and important venture into the landscape of online learning, the research behind many of its published practices was narrowly focused on how to achieve measurable outcomes most efficiently. The Hybrid Initiative was also relatively short-lived. With the departure of Executive Vice Chancellor and University Provost Alexandra Logue in 2014, the Hybrid Initiative lost support from the central administration and practices have not been reported in quite some time. Institutional gravity seems to be pulling resources toward efficiency-driven models that can easily determine quantitative outcomes. Creating new online and hybrid courses without the necessary resources to balance the need for efficiency with innovation and pedagogical training creates an environment where faculty are unprepared to manage technologically sophisticated courses that are instructionally underdeveloped and lacking in rigor and complexity.
The Hybrid Initiative site, housed in CUNY’s Academic Commons, points instructors to a study that illustrates how to align assessment with learning outcomes in evidence-based evaluation (Crespo, et al, 2010, p. 1). While the study defines skills-based assessment strategies that are effectively aligned with course and institutional outcomes, its central focus is driven by outcomes assessment. The study indicates that assessment should “validate the achievement of the learning outcomes” after students complete a “unit of learning” (Crespo, et al, 2010, p. 1-2). Many, if not most, of the online and hybrid courses in CUNY are delivered through the Blackboard Learning Management System (LMS). As the primary LMS for CUNY, the Hybrid Initiative reports that the university offers “thousands of courses” through the system (Hybrid Initiative, 2014). Blackboard works seamlessly and in conjunction with assessment-based learning practices. One example from Bronx Community College indicates that each online or hybrid course is administratively evaluated using a rubric. One of the domains in the rubric is Assessment. For the course to receive an exemplary rating, the instructor must employ “assessment strategies frequently, and in multiple formats to measure content knowledge and skills” (Bronx Community College, 2012, p. 2). While Blackboard offers a wealth of opportunity for the mass-production of efficient web-driven courses, the system’s limited functionality constricts innovation.
CUNY requested $535,000 for the School of Professional Studies (SPS) in the 2013-2014 budget. In comparison to the over ten million dollars needed to support the Borough of Manhattan Community College, SPS provides a range of professional courses online to over two thousand students at a fraction of the cost to the University (The City University of New York, 2014, p. 46). SPS’s focus is two-fold: to offer practical degree programs for working New Yorkers, and to offer these programs in an efficient manner. While the efficiency model is useful to students with busy schedules for whom face-to-face courses would be impossible, the institution relies almost solely on Blackboard to provide instruction. SPS offers its faculty a range of paid opportunities for professional development in areas such as course design, test generation, grade book operations, and pedagogical practices. While these opportunities are illustrative of the types of activities that should be happening around online and hybrid course design across the institution, SPS remains committed to using Blackboard, rather than exploring diverse opportunities for online and hybrid course delivery. SPS would be an ideal incubator to pilot alternative modes of delivery that attend to interactivity and experiential learning and would also serve as a flagship for the University in online and hybrid pedagogical research. Because of CUNY’s relationship with and reliance on Blackboard as the central mode of delivery for online courses, the University continues to limit its ability to innovate curriculum and instruction. It also lacks scholarship that examines alternative forms of online instruction, which would be valuable for faculty and students.
The analysis and critique in this brief indicate that CUNY has a number of competing priorities. The university’s attempt to engage in reform-based practices need not change. CUNY’s position as the leading urban public institution in the nation is significant and, hence, its commitment to innovative research should not be undermined. In fact, the university has the potential to become a leader in innovative, 21st century, technology-driven teaching and learning practices that support critical research and collaborative thought. The following recommendations to CUNY describe how a shift in University-wide policies can result in restoring the value of collaborative faculty and student research to the institution.
Recommendation #1: Provide accurate reporting of data on courses and students enrolled in online and hybrid courses through the Office of Institutional Research and within the PMP.
The Office of Institutional Research has an extensive online warehouse of University data ranging from graduation rates to demographics. It appears that countless variables are accounted for, analyzed, and publicly reported. What is absent, however, are data on the number of courses offered and the number of students enrolled in online or partially online classes across the network of campuses. A number of logistical changes can allow for these data to be more easily and accurately aggregated.
The taxonomical structure used to determine types of online and hybrid courses within CUNYfirst must change in order to remove the lack of clarity that presently exists around current practices used from campus to campus. There must be a common language across the University for how online and hybrid courses are represented within the system. The categories of face-to-face, hybrid, fully online, partially online, and web-enhanced have varying definitions within each campus, a pattern evident in the PMP reports. The metrics in the PMP have been altered a number of times and these alterations demonstrated that inaccurate reporting of online and hybrid courses is symptomatic on many campuses. CUNY administrators have worked to realign the metrics of the PMP but they have done little to solve the actual dilemma.
The Chief Information Office for the City University of New York should work with the campuses to establish a common language for reporting the number of students enrolled in online or hybrid courses. This language should be consistent with the definitions listed in the body of research that closely examines the effectiveness of online learning. Codes such as online, hybrid, and blended are common within the academy and CUNYfirst should be restructured to reflect these easy to understand names. Once this framework is established and operationalized across all campuses, the Office of Institutional Research should then be able to freely access and report on the number of courses offered and the number of students enrolled in online and hybrid courses. These numbers should then align with those self-reported by campus administrators in the PMP. Improving the accuracy of the data reporting the extent of online and hybrid course usage will enable campuses to better identify how institutional resources can be used to improve the impact of instructional technology on learning and on educational outcomes.
Recommendation #2: Launch a competitive grant program for faculty to research integrative and innovative online and hybrid methodologies and practices.
The 2012-2013 Office of Academic Affairs Progress Report clearly illustrated CUNY’s commitment to expanding online and hybrid courses in order to increase enrollment and graduation rates. In response to the PMP, the report stated that standards “have been helpful but insufficient” in identifying the exact number of FTEs in online or hybrid courses (City University of New York Office of Academic Affairs, n.d.). One recommendation offered in the report is that the PMP should include specifications for “accuracy of reporting” (City University of New York Office of Academic Affairs, n.d.). The same document, however, indicates that other experimental initiatives received grant funding for continued and sustained research. CUNY should again launch a competitive grant program for faculty to research integrative and innovative online and hybrid methodologies and practices.
A brief review of CUNY’s internal Community College Grant and Collaborative Incentive Research Grant from the last five award cycles yielded no evidence of scholarship in online and hybrid pedagogy. The gap in productive research in this area coupled with CUNY’s proliferation of online and hybrid courses signify a need for a specific internal grant program to promote innovative online and hybrid practices. Indeed, the University’s 2014-2015 budget request states that growth in understanding the full potential of online and hybrid learning requires deep faculty input and research and that “scarcely less obvious is the need to identify effective uses and increase access” to the resources needed for this pursuit (The City University of New York Budget Request, 2014, p. 16). While this sentiment is promising, there is little evidence at present that signals a University-wide push to encourage research and innovation in this area.
The proposed investments in technology and faculty development initiatives are significant. However, just as the grant program for the Hybrid Initiative did in 2009, the University can start with a small-scale award program that can then be scaled-up as evidence of best practices in these innovative areas become operable in departments and programs. While the preliminary grant program dedicated to launching the Hybrid Initiative was a valiant step on the part of the university to support faculty in building best practices, this grant competition must go even further. The impact of exploration must be paramount if the university is truly committed to enhancing student learning in online and hybrid courses. Multi-user virtual environments and interactive games are complex systems centered in critical thought. These systems are not meant to foster efficiency or to mass-produce pedagogy. However, they do foster experiential and complex learning integral to the mission of the university. James Paul Gee (2007) notes that in complex, interactive environments like games, “learning involves taking on and playing with identities in such a way that the learning has real choices” (p. 64), an academic habit that is essential to the types of problem-solving skills education reformers say students will desperately need.
The university should also consider the implications of researching this type of integrative and innovative technology to support CUNY’s online and hybrid initiative. At present, online and hybrid courses simply replicate the classroom experience. Blackboard, as a modular Learning Management System, is a simulation of the content- and assessment- driven paradigm of yesterday. This grant program would encourage faculty to explore online environments that are experiential in nature to revolutionize the learning experience rather than replicating traditional experiences.
David Kolb (1984) defines experiential learning as placing an “emphasis on process” rather than on “behavioral outcomes” (p. 26). Kolb goes on to say that “ideas are not fixed and immutable elements of thought but are formed and re-formed through experience” (1984, p. 26). With a focus on experiential learning University-wide, the institution’s online and hybrid initiatives should reflect a similar pedagogical approach. Interactive platforms give students the opportunity to explore, think critically about their virtual surroundings, and interact with individuals who are not necessarily part of the course. Interactivity and virtual relationships will be necessary to the 21st century workforce skills that students need to learn in higher education. While continuing its investment in Blackboard as an LMS, the University must also consider non-traditional learning environments to enrich both research and teaching.
Recommendation #3: Include research and innovation in online and hybrid environments as part of the Performance Management Process for the 2015-2016 Academic Year.
College presidents and executive staff should be held accountable for research in innovative scholarship centered on online and hybrid learning. It is recommended that the PMP be revised in the 2015-2016 Academic Year to reflect this endeavor.
The percentage of faculty engaged in online and hybrid research is a vital addition to the Performance Management Process. While the merit-based reward system for chief executives of the university need not be dissolved, college presidents should also be incentivized to provide resources to faculty who have a desire to engage in innovative online and hybrid research. Since campuses are expected to increase the ratio of FTEs to online and hybrid courses, college presidents should subsequently support research in this area. Faculty can teach in online and hybrid environments, but they should also have release time set aside at each institution to work collaboratively in identifying innovative software solutions, developing curricula, and piloting experimental online and hybrid courses. Campus presidents should work with provosts to devise a substantial amount of release time dependent upon the size of their campus.
One way to operationalize these recommendations is to use the model of professional development designed and employed by the School of Professional Studies, which can be modified and extended across all campuses to bring faculty together to identify and pilot best practices within their departments and programs. SPS offers a 2-week workshop for any CUNY faculty member who is planning to teach in an online environment. The workshops are delivered online and address “design issues, pedagogical approaches to teaching online and hybrid courses, as well as organization and management for an online class” (School of Professional Studies, n.d.). As a preliminary step, campuses might consider making this experience mandatory for faculty assigned to teach in online or hybrid environments. Since this program is delivered in an online setting, enrollment capacity is not an issue, and the University’s main commitment would come in the form of compensating faculty for their time in taking the course.
SPS also offers Faculty Peer Mentoring, linking seasoned online instructors with novices or those seeking additional support in developing their courses. Campus administrators should be encouraged through the PMP process to support faculty peer mentoring relationships as an innovative form of collaborative research. Georgia Southern University defines this practice as the Scholarship of Teaching and Learning. This collaborative endeavor “engages individuals in designing, conducting, and publishing research on teaching and learning” (Georgia Southern University, n.d.). With intentional and consistent institutional support, university faculty will have the opportunity to learn from each other and to develop courses that meet the needs of a range of CUNY students.
Educational technology is an attractive and marketable investment area for private enterprise. In the past five years, American public education has seen rising interest among companies looking to capitalize on the national push to increase accountability and gather more data on student learning to minimize risk and increase graduation rates. As public universities increase enrollments to absorb a growing number of high school graduates, outsourcing the development of online courses may seem a cost-efficient approach. However, as illustrated in the failure of the bill in the California legislature, universities may prefer to keep course development within the confines of the academy. Moreover, trends within the City University of New York underscores the need to unify institutional priorities in terms of how online learning is supported and developed.
At present, accountability measures are not supported by accurate data and misrepresent initiatives that are taking place across the network of campuses. Furthermore, institutional support of the development of online and hybrid courses changes with administrative turnover. Yet, on campuses like the School of Professional Studies, there are examples of how to approach course development and pedagogy through sophisticated professional development programs and peer mentoring. With such an extensively rich history of scholarship and practice in areas of online and hybrid learning, CUNY has an obligation to provide high-quality digital courses to students. The School of Professional Studies notes that it “is committed to CUNY’s core mission of access with excellence, with programs and services that add new dimensions” to the lives of the students of New York City (School of Professional Studies, n.d.).
Currently, this rhetorical dedication to equity and social justice is stymied by inefficient reporting systems, a lack of commitment to research in emerging pedagogies, and inconsistent institutional support of research ventures that focus on how to best serve students online and through technology. The City University of New York must redirect its attention to online learning in a manner that reflects ambitious and evidence-driven projects to innovate within higher education. The above recommendations provide frameworks for how the University can approach the next stage of research in online and hybrid course development.
“Five Year Trends in Enrollement” (2013). Five Year Trends in Total Fall Enrollment. Retrieved from http://www.cuny.edu/irdatabook/rpts2_AY_current/ENRL_0011_5YR_FA_TRND_COL-KB.pdf.Bronx Community College (CUNY). (2012). Course Site Review Evaluation Rubric. Retrieved from https://hybrid.commons.gc.cuny.edu/files/2012/09/BCC-Hybrid-Course-Site-Review-Evaluation-Rubric.pdf.
City University of New York Office of Academic Affairs. (n.d.). 2012-2013 progress reports. Retrieved from https://www.cuny.edu/about/administration/offices/aa/goals/archive/2012-2013/2012-2013progressreports.html.
Crespo, R., Kloos, C., Derntl, M., Neumann, S., Oberhuemer, P., Najjar, J., & Simon, B. (2010).Proceedings from EDUCON ’10: Aligning assessment with learning outcomes in outcomes-based education. Madrid, Spain.
CUNY Office of Institutional Research. (2014). University Performance Management Process. Retrieved from http://www.cuny.edu/about/administration/offices/ira/ir/data-book/current/accountability/PMPUniversityReport_2013-14_Final.pdf.
Gee, J.P. (2007). What Video Games Have to Teach Us About Learning and Literacy. New York: Palgrave MacMillan.
Georgia Southern University (n.d.). SoTL Commons Conference. Retrieved from http://academics.georgiasouthern.edu/ce/conferences/sotlcommons/.
Goldstein, M. (2013). The Future of Higher Education. Speech presented at the CUNY Financial Management Conference, New York, NY.
Hybrid Initiative (n.d.). Retrieved from https://www.cuny.edu/academics/initiatives/academictechnology/about/hybridinitiative.html.
Kolb, D. (1984). Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.
McDonnell, S., Chellam, C., Littman, C., & Crook, D. (2013). Implementing Value-Added Measures at the City University of New York [White Paper]. Retrieved from http://www.cuny.edu/about/administration/offices/ira/opr/papers/accountability_measures.pdf.
Milliken, J. (2014). Plans for a 21st Century CUNY. Speech presented at the Association for a Better New York, New York, NY.
Otte, G., Pecorino, P., & Picciano, A. (2001). Distance Learning and CUNY: A Broad Overview.
Retrieved from http://www.qcc.cuny.edu/socialsciences/ppecorino/psc_dlart.html.
Picciano, A. (1998). Developing an asynchronous course model at a large, urban university. Journal of Asynchronous Learning 2(1), 1-14.
Rivard, R. (2013). MOOC bill dead for now. Inside Higher Education. Retrieved from https://www.insidehighered.com/news/2013/08/01/controversial-california-bill-outsource-student-learning-dead-until-2014-or-later.
Rothbard, Richard (1994). Proceedings from CAUSE Annual Conference: Designing and Implementing the CUNY Open Systems Center. Orlando, FL.
School of Professional Studies (n.d.). About the School of Professional Studies. Retrieved from http://sps.cuny.edu/about/.
School of Professional Studies (n.d.). Faculty Development Workshops. Retrieved http://sps.cuny.edu/ofdit/programs/workshops.html.
The City University of New York. (2014). 2014-2015 budget request. New York: The City University of New York.
The City University Performance Goals and Targets 2013-2014 (n.d.). Retreived from http://www.cuny.edu/about/administration/chancellor/performance-goals/university_pmp_goals_and_targets_2013-14_final.pdf.
Wagner, Colette. (1994). Proceedings from CAUSE Annual Conference: Designing and Implementing the CUNY Open Systems Center. Orlando, FL. 1-30.