00:00:00
Dive Deeper with Human Capital Research Corporation: Integrating Slate Systems, Analytics and Practice: The Key to HCRC’s Enrollment Strategy
Hello and welcome to the dive deeper series with Human Capital Research Corporation, where we're going to be talking about integrating slate systems, analytics, and practice. We're joined today by.
Luke Robinson
02:00:24 PM
Hello from Calvin University
Misty Moye
02:00:34 PM
Hello from Boulder!
Chris Browning, Rob David and Brian. And I'm gonna let them all introduce themselves a little bit more and sort of just to get started here. A little bit of housekeeping. This webinar is being recorded and will be made available for viewing. Closed captioning is available and you can enable it right in the top hand corner. Or maybe it's that way over there. Clicking on the little CC button full screen is also available. It can click that right? A couple buttons over with the arrows. That sort of go this way and that. Should you need to re sync your audio and video.
Blake Vawter
02:00:55 PM
Go Beavs!
Steven Thai
02:01:03 PM
Hi :)
Linda Czirr
02:01:19 PM
Hello from Little Rock!
Please refresh the share window at anytime. Questions can be posted right to the chat and these four gentlemen will be sort of moderating him as they come through, so feel free to ask whatever questions you have and the chat may be turned off by clicking the chat icon at the top right corner of the share window. If you're not interested in looking at the chat and you want to take advantage of more of the screen. So without further ado, I'm going to hand it off and take it away.
OK, well thank you very much Nick. Appreciate that. Welcome everyone to the web and are this is our first webinar human capital as we are a proud to be a new platinum level partner with with technicians. I'm David Kalsbeek consultant with human capital research and I'm pleased to introduce three of my colleagues who are with us today. Brand zucker. Who's the president and founder of Human Capital Research? Chris Browning, is our executive Director of Slate Services.
And Rob Galarza is our director of Slate services.
For 30 years, human capital research has been a leader in bringing robust analytic and Research Services to higher education in many facets of strategic enrollment management and our recent acquisition of CBRG of the boutique Slate Focus systems development consultancy brought Chris and Rob to us to expand and deepen our work with our client colleges in integrating analytics with systems to improve practice. And it is this integration.
Of systems analysis and practice that is our primary focus at human capital and the source of our success with our with our clients. Ensuring that analysis isn't just someone off. Exercise done episodically, but is rather fully integrated in the day today. System support for enrollment management practice is the key to sustainable success and the purpose of today's webinar is to offer some simple illustrations of that kind of integration.
The by way of introduction, let me offer just a few brief observations as context advanced the slide, Chris.
First time I'm confident that one challenge you are all facing is the diminishing return realized through what is the prevailing practice in new student recruitment? What we see across the full spectrum of colleges and universities, with very few exceptions, is clear evidence of those diminished returns reflected in large part in the increasing number of applications, declining yield rates, rising discount rates in a nutshell, admissions staff are peddling faster and faster just to stay in place, and they're desperately seeking ways to.
Kathryn Kleeman
02:03:45 PM
Hello from Springfield, IL
Achieve greater return on their investment of human and financial resources, or at least they that's what they say they want. In reality, though, we see many just continue to feed the beast, so to speak. Buying more names, sending more communications, reaching almost blindly into new markets, peddling faster and faster, and building volume at the top of the funnel and falling further behind.
James Miller
02:03:59 PM
Greetings from your Seattle University!
Glenn Clark
02:04:17 PM
Greetings from Drake University!
Rob Galarza
02:04:17 PM
Welcome all! Good to see everyone. Please feel free to throw any questions you have in the chat, and we'll either answer them at the end of today's webinar, or we'll reach out to you with answers after we wrap up.
This the second observation is that few colleges have additional resources at their disposal to underrate their recruitment efforts. The management mantra for most of you is reallocation, not new investment. Few admissions officers have the resources to keep up with the ever increasing expectations for improved outcomes and the need to reallocate resources in in the most cost effective manner is the name of the game. Toward that end, you can only realize better outcomes by focusing on improved productivity.
Through a process of prioritization, it's easier said than done, but it's a first order challenge that we all face.
Jonnette Fair
02:04:28 PM
Hello from Atlanta, Ga...Mercer University!
Lisa Kress
02:04:56 PM
Rock Chalk from the University of Kansas!
The third observation is what we all commonly know is the 8020 rule, recognizing how typical it is in business that 80% of sales comes from 20% of clients or the 20% of products typically account for 80% of revenue, or that generally 80% of our results come from 20% of our efforts. This so called paradeau principle certainly applies to college recruitment strategies and time and time. Again, we find that 80% of colleges.
Applicants or applications and enrollment.
Comes from 20% of its overall high school or geographic draw. We therefore find that achieving the highest level of effort, optimization and reversing this relentless slide of declining productivity can be and almost always is achieved by using this principle to focus and refine the recruitment effort.
4th
Every College University says it's special. We agree. We just spell it a little differently. Colleges are spatial. They occupy a place. They're situated in a place with the exception of a few fully online institutions and some truly national schools. Geography matters. Regionality matters. Colleges draw students from a concerted geography and their market strategies and their market opportunities are tide to geography and the age old marketing notion of the friction of distance.
Is almost always in play in deciding the breadth and the reach of a recruitment strategy. Our observation is that most colleges fail to fully capitalize on integrating data on market geography into their recruitment, planning and analysis and practice.
5th observation in order to really capitalize on such analytic intelligence and insight about market geography, that analysis has to be integrated in this late systems in order to make it actionable. An analysis of the impact of market geography that sits on the shelf or sits in some PowerPoint deck and is not melted into the systems that staff use in their daily recruitment efforts is not sufficient to improve practice.
Improvements in efficiency and effectiveness of practice can only be gained by integrating analysis into our underlying systems and slate as the perfect platform for them.
And finally, by way of initial observation, it's this.
In all of these efforts, your mileage will vary.
Each college really does have distinct and sometimes unique challenges and opportunities, and the outcomes and impact of market analysis does vary. It's always valuable that to varying degrees, but we also know that the most significant variable and how effectively market analysis improves recruitment practice.
Lies not in the analysis per say, but rather in the capacity and the competency of each colleges admissions team in implementing and executing data driven practice. Bottom line, your mileage will vary in these efforts primarily because of the way you drive the car, and that's why. Bottom line, we don't offer boilerplate solutions to our partners. We have to customize and tailor slate solutions to the distinct capacities and needs of our clients as as that naturally needs needs to be.
So what we're going to show today is a way by which we support colleges and the geographic Geo Legacy High school based targeting and scoring of market opportunities and integrating that with slate. So we're going to start with Brian walking us through some of the analytic foundation and then have Chris and Rob show us how how this work can be surfaced in the on the slate platform, Brian.
Hi hi folks, uhm when we talk about integration and in a context of either our researchers Slade systems, we of course draw to a great extent on student unit record data but
part of integration and integrative thinking, particularly if we're uncertain about the future, we want to turn to as many sources of insight as we can. And if we go to the next slide, one of those is survey based research, which is a meaningful, important organizational competency. It's it should be integral to the practice as well as mastery of transactional data.
As well as public domain information, putting all these pieces to work as a bit more context sort of building on what David has described and gyne trying to get a flavor for the current state of affairs, we're looking at a few exhibits here which are drawn from a survey that we do with our clients. So this is a caravan survey involving about 80 colleges and universities.
Uh, the sample is slightly more than 90,000 completed interviews. These are just a smattering of questions that we've pulled as if you will in some ways indicative or emblematic of what some of the dynamics are in the market today. We're not going to belabor it, but it really helps set the stage for why work of this nature is so important and top slide on the left just simply.
Acknowledging the sheer number of applications that we're dealing with, that app volume grows year after year, after year, and this particular exhibit just shows a relationship between the average number of applications and the market position of a school, whether it is a highly selective, a strong, deep benched high graduation rate school or more of a middle market school, you can see.
A pretty well behaved relationship. That's a first aspect of what we're all dealing with is the sheer volume, which in no small part we have brought on ourselves by going out and being ever more aggressive and shotgun. If you will. In recruitment, the exhibit to the right shows something that really began with the repeal of the rules of professional practice, binac yak and a rising incidence of students.
To make multiple deposits at this point in time, we're up in the neighborhood of one and eleven students are producing more than one deposit and to the right of that just making note how much the world has changed in the last couple of years. Accelerated by the pandemic. But heading in this direction of an increasingly virtual way of engaging and raising the question. This, in this particular case.
Showing the number of students who indicated that they were greatly influenced very or extremely influenced in their final choice decision by what they saw or learned through YouTube or Instagram and to some extent up those influences, are outside your direct control, which is also part of a A market reality. Lower right? We're just making note of the degree to which families now.
Are appealing their financial aid awards, or, increasingly, are being solicited with adjustments to their aid packages that they didn't even ask for. And finally, the short of it presented in the box plot on the lower left in this instance, cut by the sceptile of number of applications students submitted, so each representing about a seventh of the total respondent population.
And contrast ING of those students who enrolled with our client versus those who don't, and in short, this exhibit showing the proportion of students, weather metrics or Nah metrics who are telling us. Telling the survey that you're sending me too much stuff. So the question is how all these pieces interrelate and how they come together in what de facto is a schools recruitment strategy and weather?
This massive noise that we sort of deal with there are potentially more disciplined, structured, organized approaches that we could take to help manage the allocation of scarce resources. Towards that end, we're going to introduce you now to a basic analytic framework. This just sort of presents the scaffolding. There, of course, is a lot more detail behind the curtain.
Happy to share that detail. We believe strongly in open architecture with our client, but as a starting point and how we take what some might even view increasingly as a kind of chaos and certainly an uphill climb here, here then is an approach for discerning structure.
First off, our focus really is not just on generating apps that equate to soft apps and more noise, but rather really apps of students who are potentially a better fit, engaged, qualified and in the same breath understanding conversion productivity that might be. In other words, a return on investment that is significant.
Positive to begin and perhaps a different orientation, at least for some than than usual. In this work we are making the high school or the community the micro area. The subject of our attention. So in addition to studying individual students and their behavior, we have established frameworks that really glean a great deal of insight by studying.
The high school, as the unit of analysis. Second to discern these patterns to make sense of the dynamics, one has to look over multiple recruitment cycles. This is not something you could glean in a single cycle or even two or three years. This is stepping back and seeing the forest for the trees, so to speak, and it's over these longer periods of time that we begin to really understand distinct behavior.
Where is school has brand strength where there are personal connections that always benefit this process and and that's the second kind of precept. A third is we're interested in the productivity and the conversion behavior. There are profound differences and those immediately manifest under this framework. We can readily see dramatic differences, for example, in the ratio of names bought.
And the students urge to the actual number of matriculant at feeder high schools. Those ratios might be in a neighborhood of literally as low as 20 to 140 to 180 to one. And when we get into non feeder markets those ratios can loom 1000 to 110 thousand to one or even Infinity because if you didn't matriculate anybody, that's what you get. So it's an eye on that conversion.
Productivity also a consideration of not just that productivity, but who are we drawing? What are the characteristics of the students? What are the characteristics of the territory and based on those characteristics we begin to prioritize so it's not priority based purely on the basis of productivity. Important as it is, it also has to relate to simpatico traits or.
Attributes that you consider of importance in shaping your class. We also have to understand all the other channels. All the other treatments and interactions that have a bearing on the outcome. All conduits are not created equal and there are an ocean of different channels that bring applicants to our doors. Understanding each of those channels and the role they play. Their respective contribution of conversion.
Is an integral part of this, so we're trying to take a more holistic, integrated view. UM, lastly, we are bringing all of those characteristics in tandem with the actual student interactions. The actual case level clicks and opens, and seconds on a URL. All of those micro exchanges in the right hands manifest as robust signaling.
Information, but it's really by bringing these pieces together. The net result of employing a framework like this is we can in very meaningful ways establish target geography. What we'll call concerted geography. The idea that you can't be everywhere unless you have infinite resources, you have to choose. Arguably for any college or university in the country out of the thirty 40,000 high schools.
Glenn Clark
02:18:38 PM
It was mentioned these analytics were developed (and had to be) over multiple cycles. Has (and how) COVID changed these any of these signaling elements? or is it too early to tell based on the multiple cycle nature of the data?
There is a much shorter list of high schools that are going to be much more important to that. College is future, while where are they? Who are they and on what basis? Do we identify those were also in this process able to assign a likelihood of moving from one step in the process to the next. Whether it's from surge to responder, responder to app app to accept, accept to enroll, and it's those probabilities.
Of conversion that give us the means through which to triage, target resources and assess the returns on different tactics and strategies that we've employed in the first place.
Lastly, when you step back and you look at the end of the cycle and you bring this information to bear, you have an opportunity to understand much more about your return on investment and to set course for the future. So this is just sort of walking through a series of approaches, or rubrics or structures, and if we go to the next slide we begin to have a kind of a more concrete illustration.
So in this example.
Brendon Troy
02:20:00 PM
I wonder if the market development framework could be used for graduate admissions with (undergraduate or last-attended) post-secondary institution as the primary unit of analysis, or whether certain secondary/contextual data points would not be available with segmentation by that unit?
We under this idea about making the high school the subject of our attention, we are engaging in a classification of high schools based on the number of years any given high school in the country has had a relationship with the college. As in applicants accepts and roles, you could start earlier in the funnel as well. The volume of transactions was it 1 app one year or was it multiple apps?
In a single year, the patterns of conversion moving from app to accept to enroll, and the trajectory over various intervals of time. Understanding whether the pattern HERE is rising, stable or falling, and with the right math behind this the right rubrics, we bring this together in a sort of composite performance index that to a remarkable extent helps us to separate the wheat.
From the chaff to discern out of all the high schools, or all the students represented by all the high schools, what share of the high schools account for one share of the enrollment? So Dave spoke of of the paradeau principle in this particular instance, we're getting out of a 20% of the pool, 87% of the enrolled population, and the real litmus test is seeing consistently.
That same result, year after year after year purely is a function of this classification system. This immediately can be put to work, for example, to inform a surge by and two from the get go improve productivity in search.
The real strategy, however, is not just to keep going back to the places where you have students, but rather to build and enrich that stock of feeder high schools. So part of the strategy here is if I can increase the number of these high production high schools. I'm not just bringing in next years class, but I'm actually deepening the foundation upon which to build future cohorts as well.
That's how ideas that are as much about cultivation as direct marketing. And when we say cultivation, we mean a relationship with the community that goes beyond the one off interactions you have with students in a single recruitment cycle. That's bigger than admissions that involves other divisions of a college as well. When we think deeper about target market geography and Dwyer, that territory should be in what high schools we do go into.
In addition to the productivity in this idea around history, volume conversion trajectory, we're now bringing a bear. The other attributes, and in a very up to a great extent. This is really in the eye of the beholder. The premium that any given school has on certain student characteristics you may be wanting to build greater diversity. You may be wanting to find families who are able better able to pay the full freight.
You may want to leverage the presence of alumni you may be attuned to. A proximity to market. There are many attributes that we might consider, including aspects of the underlying demography. If I'm going to hitch my wagon to a market and it takes periods of time to cultivate, is this market rising or falling for the types of students I'm after? So in this process we are bringing these other criteria to bear.
We're understanding how these criteria work in conjunction with those feeder patterns to understand conversion and out of this process, we now have a much more holistic system for ranking territory. When we look as a result of that, and again that ranking system is as varied as the schools themselves, you could introduce almost any criteria. Having gone through a rating process of that nature.
You're now in a position to rank every territory in the United States. In this domestic target market exercise. This is an illustration. This is just one of many ways this information could be presented, but it effectively for a given institution, presents a structure for organizing the markets that goes well beyond the conventions of simply calling territory first tier, second tier, third tier. Because you have these underlying.
Dynamics, which have now been taken into consideration in an output like this, we're making a unit of analysis in the larger geography, a metropolitan area as opposed to an individual state, and this is a hierarchical structure, meaning from the metro area we can drill down and get anywhere from down to the individual block group or the ZIP code, and certainly the high school catchment area.
Luke Robinson
02:25:41 PM
Quick question, what's HVCT?
All the way up to the larger Geo territory definitions, and So what we're looking at in this one picture of a market ranking behind this page, if you will, is the next layer and the layer behind that eventually taking us down to the high school ceeb and behind that to the individual students in this year's cycle behind that seat. So it is a nested structure, and that same information.
Those scoring elements. Those characteristics come along throughout each phase of of that journey. If we go from here to the next exhibit, it's just an illustration of how we move from having established target geography based on productivity and desired attributes. We're now beginning to ask ourselves where do the resources go? Are the resources really?
David Kalsbeek
02:26:29 PM
HVCT - is abbreviation of 4 foundational defining elements of a CEEB --
Consistently going into those target definitions? Or are we moving resources outside of the target geography? That in fact should be reallocated? Whether it's in the old World, Pre pandemic, a high school visit, whether it's a search by whether it's targeted outreach, and a calling campaign, whether it's direct mail in any form, understanding those targets and what expected return on conversion.
Looks like helps to direct those resources to their highest return. So if we step back and you could go to the next slide. Here you begin to have a kind of general schematic for thinking about the underlying database architecture to help inform your strategy and your day-to-day practice. And on the right side of this exhibit we have the few elements.
David Kalsbeek
02:27:25 PM
its HISTORY of apps; its VOLUME of apps, its CONVERSION rate and its TRAJECTORY over time.
I've just made a reference to that history volume conversion trajectory. We have those other criteria that are being brought to bear and taken together. They really help provide a kind of underlying structure and then it is further informed by identifying and building into that feed. What are those points of origin? Where did that app come from in the first place? Was that surge? Was it hopson?
Was it raised me CapEx, a CBO, etc. Each of those conduits then becomes part of this information set and then we layer on top all those day-to-day, very dynamic micro exchanges that really in their own right are revealed behavior. There are manifest of interest, but again, part of the key here is understanding how to aggregate, structure, normalize those.
Luke Robinson
02:28:31 PM
Thank you, David. I see it now in slide #9.
Data so that they become reliable contributing factors in a predictive model. And then lastly, you have the student attributes themselves so that you are not just sitting waiting for apps to fall, but you actually have a hand in shaping the composition of your applicant pool. All of those arrows are pointing to the center where the repository is the slate system.
Coupled with the analytics that translate that into dynamic scoring to help us manage our resources. And by the end of the cycle we're taking that stock and we're bringing a new perspective which is really stepping back. Looking at it in a more structured approach to understand where the highest returns are, where the returns are diminishing and where we can reallocate and plan for the go forward.
So that in a nutshell, is the structure that we're bringing to help inform market development.
And then as we move it into Slate, it's putting those analytics to work. So it's putting that analytical capacity all these different models and putting them in a construct within your slate environment. So whether you're the VP of enrollment or you're an admissions counselor, you're getting actionable things out of this analytic work. And it's custom to your institution and it's all multi layered and interconnected, just like everything else in slate. So we have these different layers of geographic dashboards.
Going from the unit student level to the high school and up to higher level geography like a metropolitan area and each one will have its own set of scores that are easily displayed and find four folks doing various tests. Whether it's the admissions counselor looking at the student and whether they should invest the time in having personalized communications, text messaging with them, or it's the VP of enrollment saying OK, I need to reassess where I'm spending dollars.
Search or staff travel. Looking at a more metro or collection of larger geography, you see a few examples here. I'm going to share my screen just so we can do a little live view.
And starting the Rob at our our test case here you have right on his dashboard the composite so that center of that diagram this actual scoring the conversion scoring. It takes all this other analytic work and taking the history, taking the geography, taking the student interaction and combining combining it into easy to understand score for staff and what's the likelihood of him converting to inquiry to the application stage or ultimately?
Yelling at the institution, then we have some of the other scoring models here. The taking into account the level of engagement so that generative communication, that interaction in the slate system, or affinity that takes the other attributes of the student into account. And the Geo demography putting it in context of where this student is coming from.
For your campus and like I said, it's all layered so we could go up to.
The high school and now we have that same level of analytics to actually take a look at what that high school is in that context. In this case, we're looking at the HP CP score that Power score, which is taking that plus then the power to shape through analytics a little bit more. Whether this should be a strategic target or not, and then some of the components that make that up and again associated geographic area. So through related dashboards through entities you can.
Keep on going.
And now we're at the metropolitan area. So now we're looking at at Hartford and it's Marketeering position as well as the same level of analytics and component scores here. And we can even go out and connect to a another data visualization tool like Tableau, and now have that slight data common appear in Tableau, and you have the power score you have that HBCT score. You could do any level of data analysis again custom to what your goals are on campus.
And see very clearly. OK, this is the Hartford area. This is what the power score that HBCT sport is for the Hartford area or even go out nationally and now take a collective look at a collection of these.
Glenn Clark
02:33:26 PM
Could this be leveraged as an Overlay in Voyager?
A core based statistical areas to further define at a more strategic level what you want to do to shape enrollment at the institution. So it's through these slate tools and then the power of the analysis. The power of the data, science and analytics that brings this altogether right within Slate. So it's it's right in your slate instance to be able to give you these actual steps from the strategic, the tactical, all the way down to the execution level.
And this is all about the evolution of practice which David is going to speak a little bit more to.
David, you're muted.
Thank you, want to make sure we leave some time for questions and we see a number of them are rolling in in the in the chat, so we address that. But before we wrap up, we really the application of this kind of information. As Chris noted, you know is is is clear in terms of the decisions and the choices and the policies and the strategies that are developed either at the with the Vice President for enrollment and again using some of the scoring 2 to shape and frame strategy. Or better yet to help inform.
Others at the at the campus leadership board. About the market dynamics and the and the underlying and the regionality and the factors that shape enrollment outcomes and then the application and and implications work their way through the organization from the admissions director is using and they scoring in allocating staff time and travel schedules down to the admission staff member who's who's looking at this information at the student level in their own recruitment effort. Implications for the marketing team and how they design.
Johnna Watson
02:34:56 PM
@Brendon Troy
Johnna Watson
02:35:02 PM
Yes, I'm wondering the same.
Communication strategies targeting again to try and prove productivity around some of these scoring in the messaging around around that. Overall, what we're what we're seeing here is. I started by noting the diminishing returns on prevailing practice, and I think what we're what we're doing here. What we're addressing here, with certainly with our clients and our partners, is is facilitating what we see as more of the emergent practice in in student recruitment and enrollment, development and market development.
This evolution certainly includes this shift from focusing on on lead generation to a focus more on lead qualification. This isn't at all a new notion we were talking about this in the in the business for for a long time, but what we have with Slate is that the capacity to really make this a defining part of our practice in the admissions team. Upbringing, focus, spring prioritization and improving the productivity of our efforts. And again addressing that phenomenon of of trying to trying to bring to a.
And bring in the pedaling faster and faster just to stay in place and really focusing on it on really optimizing our time on task and having it through the qualification of leads.
We see a shift in the in the focus from high volume where schools are forcing more and more and more into the top of the funnel with the hope that more and more and more will come out at the bottom. It doesn't work that way. We don't have gravity in our favor most days. What we're trying to do is is to moderate the manic pursuit of greater volume through better targeting, prioritization, scoring to improve overall productivity, and taking the kind of analysis that feeds that. Integrating into our slate platform.
Even much of the prevailing practice is is a spatial. It's not sufficiently recognizing the power of geography and Geo legacy patterns to frame strategy and to focus practice. Territory management has always been a big part of the admissions work and or improving that practice by empirically intelligently making choices, setting priorities as we allocate and reallocate staff time budget resources in our recruitment efforts.
We certainly recognize that the real game change that Slate offers as Brian described is the shift from just using static attributes in our predictive modeling of likelihood to enroll or to apply to the incorporation of more dynamic attributes. Geo scoring is a good place to start, but the real value of what we're talking about here is how we incorporate the data on the dynamic interactions that we have with perspective students.
Uh, it that are tracked in Slaton can be incorporated into into this modeling process as our communications continue to shift in this evolution of emergent practice shift from being transactional in their focus passive in their content shotgunned in their direction and generally pushing out messages. What we're trying to do is is evolve into and the emergent practice is to be for the for our efforts to be more relational and engaging in their content far more targeted.
In their direction and pulling our perspective students into greater interactivity with with with our with our admissions staff and with our institution.
Finally, it's it's not uncommon for colleges to to celebrate all of slates capacities to bring more data and more accessible data into the mix of our work, and in fact, some of what you've seen demonstrated here would might be seen just as a valuable expansion of data to be grist in the mill of enrollment practice, I think, but all of us would adamantly assert that the goal is not more data.
To the contrary, it's it's more insight at the analysis that we do is the impact of that is in how we filter and reduce data, how we integrate disparate data sources, how we distill insight, and understanding from that data, and then apply that to practice the emergent practices more informed, more integrated, more applied. Then we often find in the prevailing practice at many institutions and that bottom line, I think, is is the value that we're that we're trying to realize.
Sure.
At Human capital research and our work with our with our partners, we have a final exhibit, Brian for wrap wrap us up here before we take some questions.
Yeah, and this again is drawn from this year's admitted student review. This is again, that's.
Luke Robinson
02:39:55 PM
Last slide: https://slate--partners-technolutions-net.cdn.technolutions.net/share/slide?id=182b429a-7eaf-4f59-94ab-6f984ba157c1
Yeah, and this again is drawn from this year's admitted student review. This is again that same sample of about 90,000 of students admitted to one of our clients. By and large, just of note, these students are ultimately enrolling at schools that are comfortably in the top 40% of the market, and what we see here in this.
Rather straightforward question. What sources of information mattered most when deciding which schools to apply to for admission and the answer is up, well, all of them. They're not mutually exclusive. Students are identifying multiple critical information sources, even when the incidence of something is lower, its influence could be greater, but the real takeaway is how these various sources of information or.
Points, UM, interact with each other and to understand how they come together as a confluence to induce students not just to employ, apply, but to enroll again, one needs to have a system and a structure in order to manage that. And it's not an either or here. It's a matter of how we bring these pieces together into a framework in which we can take stock.
Of what the relationships are pre-existing, the characteristics of the students, the channels through which they arrived at our doorstep, and then ultimately the ways in which they have now engaged with us, the manifest of which comes in large part through all those interactions. That slate is capturing. And the trick is in how you take these. What are literally millions of transactions and can affectively distill them.
Show that the observations you make about the relationship between any and all of these exchanges in a prior period are still applicable. Still, bring value and insight in a forthcoming period. No small task to understand that the signaling and its association with yield last year may in fact result in different outcomes this year and question. And this is where technical comes in to really understand.
How to normalize those data, bring them to bear in ways that produce reliable estimates as you go forward. Also, just to note that that signaling in no small part is tide to how you engage in the first place, who you engage with and how you engage so that that manifest of interest is neither preordained nor random.
You have a hand in inducing that signaling, and that too is an integral part of what building a platform like this should be. So our effort in the moment was really to try and a short period of time somehow unpack or rather complex series of of frameworks and approaches for thinking about market development that are in every manner holistic and integrative, and again bring the technology.
The analytics and the practice, all of which invariably must be tailored to the individual school. There is no one size fits all, and in fact this is a constancy of purpose because it's not just that every school is different, every cycle is different as well. So I I think we're at a point where we're up to some Q&A or wherever folks want to go in this conversation.
Yeah, we've got some questions that have been coming in both during the webinar and some that we got ahead of time from some of the registrants and there were a few of those I wanted to walk through and post to the group.
Uhm?
You know this past week or the last week or so, Apple made news with some some updates to iOS and how Apple devices are going to handle the processing of opening email and the lack of insight that schools will have. Anyone will have in terms of seeing open rates because of the way that they're using proxies to kind of preload and pre open those messages. So we know that's a kind of a pending reality that's coming down the way.
As we look through deliver statistics and the message and mailing tables within Slate, you will have to account for that. But the question Brian I think maybe best guide to you. How do we know what matters? How do we? Based on all this data when it's open rates, click rates paying all the myriad ways that we mirror data points that we have. How do we know what matters? How do we?
But that's OK, question one. And then you know.
Yeah.
How do you track these things over time, especially given the COVID reality we've been in right? Pre COVID, a campus tour followed by opening emails were great predictors that students would apply, but without those tours, and even now as students opt for more web visits, the person who asked this question said they feel a little lost. So how do we define and figure out what matters?
Well, uhm.
We
and.
His face.
Well uhm, we have a method which is both looking backward and looking forward in the context of looking backward. We're very interested of course and how any transaction correlate's with a likelihood of, let's say, yield if that's the outcome. It doesn't necessarily have to just correlate with yield as in the final step in in the recruitment process.
But it could for the same difference correlate with antecedents. Other factors that are related to yield whether that, for example a robust signal that manifests through visiting campus physically, or a combination of applying for aid and visiting. We know students who engage in that manner are much more likely to make it up to the next step. The first point of reference is looking at prior cycles.
Multiple prior cycles and understanding whether in fact there is a correlation and that correlation of itself immediately provides a kind of factor weight that you can employ in which we might say not all transactions are created equal. Some have a more robust relationship with a desired outcome or an antecedent than others up. The variation is forward looking, however, where we say, let's dispense with last year entirely and let's just look.
Right now in the current cycle. So what markers exist over the recruitment cycle that are clearly steps in the right direction? And again in this instance, even if you've never had in prior years, a transaction such as a virtual interview, you can get into the slate data and discern it's it's, I think self evident that somebody who participates in a virtual interview.
Is more likely to convert then somebody who hasn't per say, but it's not just a binary flag. This is where the treatments matter and so can you get into this late day to find the URLs that are tide to the virtual interview and actually examined the duration of that interview. Was it a eight second interview or was it a 5 minute interview and you can build classes of transactions that clearly move?
And as each day unfolds over, let's say a 250 day period from September 1st to may one, you can keep building and building adjusting fine tuning. Combining those markers, and so you need not model just in a looking backward context, you can have a forward looking framework to understand which of these signals matter more than the others based on their relationship.
Two milestones or big steps along the way.
So a related question from Glen in the chat was we were talking earlier that you mentioned the analytics.
Yep.
Have been developed and have to have been developed over multiple cycles. You know, has COVID changed these signaling signaling elements? Obviously talked about the talked about the the online visit or remote visit, but you know again. Kind of looking forward. Is it too early to tell based on the multiple cycle nature of the data?
Taun Toay
02:49:14 PM
What are the most powerful data-points that you find commonly missing from files when you start working with new partners?
Uhm, they've changed profoundly. It is so dramatic. The shifts that to draw on, let's say 2018 as a frame of reference to project what might happen in 2220. Eighteen might as well be 1950. We are so far back in the past, even 2019, the dynamics were completely different. It is more fluid.
And it really requires us to attend to these ideas about forward looking instead of backward looking, but.
Tom Nicholas
02:49:41 PM
In terms of lead qualification/prospect scoring, one of the tough choices that schools often face is whether to use simple homegrown models, built with something like Slate's scoring capability, that are immediately available -- versus using a much more robust and accurate model such as HCRC's, but where scoring files are only received periodically (and thus don't allow immediate action based on scoring). Is there any thought to bridging that gap, either via some kind of regular Slate/HCRC data bridge, or via some type of simplified formula, based on the most significant factors in HCRC's model, but applied via rules/translations within Slate to provide provisional scores for records until better values arrive?
That doesn't mean we want to dismiss everything that happened in the past. 'cause there are still benchmarks. There are still of significant characteristics that are going to be positively associated, so you have to proceed on both fronts. You look backward and you look forward and then you look for the inter relationship between the two. And again, this idea of what we call inner temporal reliability. The idea that a certain behavior observed in a back period.
May in fact mean something else in the current period, and so we can never take a transaction from the past verbatim. But that doesn't mean there isn't an association. We just have to be attuned to how to temper that association, how to normalize that association, and one of the ways we do that is by always looking at that dynamic relative to a student peers at the given point in time. So there are methods.
We're doing this what we call normalizing the data that allow us to. If you will traverse this inner temporal problem, but we have to be on our toes, there isn't a version of being successful on an analytic front where you can take for granted a rigid set of algorithms and just slam based on that same recipe you were using last year or the year before. This also requires different approaches.
Modeling in which you never rely on one model. You rely on a combination of models and what is sometimes referred to as ensemble modeling. To understand the degree to which these models corroborate one another, or refute one another, we have to deal with volatility.
Great, thanks Brian. There was a question from Brendan about obviously the lens through which we are looking at this and through this initial market development framework was the seed level and specifically high schools.
The question was about whether a similar framework could be used for graduate admission where you're looking at last institution attended either undergraduate or other grad schools, or even going back to high schools, and that weather data points would be available with segmentation by that measure.
Yeah, uhm, in some ways graduate market dynamics or even more regional, more provincial than undergraduate dynamics. Clearly, you know if we're talking about funded doctoral programs, that kind of program may truly have a national draw, but in many instances, graduate markets affectively play out almost within a commuting periphery. Putting online off to the side.
For a moment, there are corollary. The feeder patterns aren't as robust as we observe at the high school level, but they they certainly do exist, and they are meaningful parameters that you can put to work on understanding those spatial dynamics. They're they're in in every respect. These tools are portable in a graduate context, but they need to be adapted accordingly.
Let me add, add a couple things to that. One of the one of the challenges when looking at graduate programs is that certainly unlike the traditional first time, full-time freshmen in the applying to more and more and more institutions as Brian illustrated in some of the data, we tend not to see that with with a significant number of graduate programs. Professional graduate programs, those decisions about where a student intends to pursue their MBA program.
To pick up their Masters degree are largely made prior to the point of application and admission, so that's the shape of that. Funnel is dramatically different in graduate programs. Nevertheless, the geography matters. Prediction predictors matter as best we can capture that to graduate level 10. Most graduate offices just tend to have a more anemic scope of data that they're collecting in that process. To build these kinds of models and.
And you and you do the best you can with with with with what you've got. And it tends to be in an area that's under invested in and make one additional comment. Again, depending on the nature of the graduate program, that is significant variable in that may not be so much their post secondary institution but but but rather their current employer as a as a as when you're looking at at predictors of enrollment or source of students. If it's a part time, business degree or program, it's often where you find that the source that matters is.
Is the place of employment what matters is the nature of the tuition, tuition policies, tuition remission policies, tuition support policies at that employer.
Certainly for the 20 years that I was at DePaul University, you know we'd have. You know that there were there were hundreds and hundreds and hundreds of alumni working. For example at Northern Trust, and that that created a very fertile ground for graduate enrollment recruitment. Given that alumni base, given the nature of programs could be targeted to that to that population, so so again, depending on the school, depending on the graduate program, employer becomes a critical part of the kind of the understanding the source.
In the market context for potential enrollment.
We got time for one more from this. The question that Tom just asked here few minutes ago. Dovetail really nicely with one of the other ones we got ahead of time.
Uhm, the gist of it won't read all of it, but these schools are facing these decisions about whether to what degree to use their own kind of in-house metrics for determining.
Brendon Troy
02:55:58 PM
Thanks for the response to my question about applications of the framework to graduate admissions (and for the overall presentation). Have a good day.
You know prospect scoring etc. Comparing that to what obviously we're doing at human capital, and one of the kind of choke points for lack of a better term that has been in the past is the frequency with which some of the data that human capital could be providing to the schools. With these different model scores.
Whether we have plans in motion to help bridge some of that, that those gaps right now, and I think Chris might be a good person to talk about what we've been doing that area.
Yeah.
Yeah, so one of the things we're working on and going to be releasing in phases to clients is using slates, SFTP, infrastructure data automation infrastructure to be able to get standardized files back and forth where given the complexity of some models it's always going to be preferential to.
Actually run those models outside of slate, but to be able to refresh them on a more regular basis using automated tools to simply have prebuilt queries to actually push files out to the SMTP and then having prebuilt source formats that will allow you to bring back those scores and that data as the model runs. Or some of our models though, they can be put directly in slate and to time you kind of alluded to. This really depends on how many different variables are in there.
Out what's going on. Their academic rank is a great example of the application side of something that could be run and we deployed directly in slate databases. Something like that conversion scores. It's taking all this different student signaling generative communication. Just the sheer data volume of those models you're going to want to run for performance issues outside of slave, but through the automations, then inflates tools for exporting and importing data. You can get it on a regular cadence.
Tom Nicholas
02:57:40 PM
Cool - thanks!
I just to add in, I'll make note that while we need to stay current with transactions as a cycle unfolds of the day-to-day micro exchanges. If you're looking at a days worth of transactions and presenting those to a model as a standalone set of independent variables.
Up that kind of construct is not really going to register with most models. Those signals would have to be really robust signals that we associate with with big recruitment events like an admitted student day in the spring and OK. So now we're basically tallying who did that at the end of the day. But in the day-to-day micro exchanges, looking day today is is really getting lost back in the noise.
And the the more appropriate question is how do we structure that? And to aggregate that and treat that and and and so the interval doesn't out of necessity have to be a daily refresh it it needs to be a periodic refresh and there need to be protocols in place so that when the refresh occurs the conditioning can happen almost instantaneously. But that's a different orientation.
Uh, because now we're really stepping back thinking about a recruitment cycle mapping out when these critical transactions should take place, informed by key milestones along the way. Decision points that have to be addressed along the way, and so it's less about real time monitoring than really pausing for a moment and doing real assessment.
Perfect alright.
I think that brings us to the end of our time. If anyone has further questions, please feel free to contact Chris or Rob directly and would be glad to glad to respond as a follow up to this web and R and take it back to Nick.
Thanks everybody, we really appreciate you taking the time to to present all this really cool, really interesting stuff. We have more in this series coming to you this week and next week and the week after that. Stay tuned for this Thursday. For where are we a dive deeper series with niche. Now you're speaking their language so thank you again to everybody that was on today. Thank you to the presenters. This was fantastic.
And I will see you all again soon.
My folks.
Misty Moye
03:00:28 PM
Thank you! What an interesting presentation!