Tag Archives: Data

Tony DePrato

Stop the Downturn: Data for Student Support

Leave a comment

By: Tony DePrato | Follow me on Twitter @tdeprato

For many years I have been involved in student support planning. As an EdTech professional, I am heavily involved in managing and using student data. Student assessment data is normally used to make lists of students that need support.

The ideal scenario, is that students get the help they need BEFORE the grade falls below the recovery level. There is normally a point in the term where the grade cannot be recovered. The mean will be too low. If the school uses a few final exams to determine the final grade, the situation is even more dire for students who have early downturns.

Here are some recommendations for making certain that you are using student data correctly, and promptly, to support those who are beginning to have unfavorable results.

Set the Bar High

I start my trend analysis at the C+ level, or “average”, level. I look for students who have a C+, and see if they had a C+ the week before. This can be done fairly quickly in a spreadsheet with live data sets.

Students who have moved from a C+ to a C, a C- to a D, etc., would all need a weekly review.

This seems tedious, but I firmly believe interventions need to happen as early as possible in the process.

Do not Assume Students are Lazy

I am often guilty of assuming a student is simply not trying hard enough, or not paying attention. I think this is a very common initial reaction to falling grades.

Every student deserves to have the benefit of the doubt. Take the time to look at least 1-2 weeks back in the grading. Look for courses they are not struggling in, and see how the assessments differ.

Most importantly, take time to engage the student. Ask them about the situation, and listen for clues. Many times teenagers seems cagey, but they simply may not be able to articulate the problem.

Check the Class Average

Class averages often hold insight into student issues. If you have a class, and the average is 80%, and the grade distribution is on a normal curve, then prepare to have many students struggling.

That bottom group of students is going to be fighting all term for a low B or high C (80%-76%). This does not mean they need extra support, but it does mean that they need to be using their time very efficiently. The margin for error, and laziness, is very low.

Also, do not jump to make the class easier. Some topics are tough, and they should be.

Convert Standards Grades to Numbers

This is an internal process. Students and parents will not see the conversion. This is not about creating a 100 point scale. This is simply a better way for administrators to quickly review data. You can use any scale you wish.

If you have only three standard’s indicators, and you are only grading against four standards, you would generate 12 data points, per student, per assessment. That is 216 data points per 18 students, per assignment.

Assigning numbers to letters, using a simple find-and-replace function, would make it possible to run common mathematical analysis.

Require Regular Comments

End of term comments are nice, but they are useless for a true support intervention process. Teachers need to be required to tag assignments at the student level when those assignments indicate a downturn.

Many administrators are often sitting in a room without the teacher trying to understand the data. Simple comments bring clarity to assessment data. This is true even in standards-based environments.

I would even argue semester and trimester comments are useless. Action needs to be swift, and data needs to be updated weekly.

Require Teachers to Update Grades Often

Obviously, without data, no action can transpire. Data needs to be updated every 5-10 school days. If a teachers gives 4 significant assessments in a month, and updates their grades only once every 4-6 weeks, how far will the grade(s) fall before an intervention can happen?

Keep in mind there is a gap between the time the issue is discovered, and the engagement with the student(s). Every day matters. Make a point to be the annoying administrator who is sending “gentle reminders” about grading and data updates.

 

 

 

 

Tony DePrato

The Maker Portfolio and University Admissions

Leave a comment

By: Tony DePrato | Follow me on Twitter @tdeprato

I am always focused on the end-game. The end-game for students is the next level after they leave K-12. Preparing students to compete and succeed is difficult. There is always a huge debate over where time should be allocated, what subjects are more important, and what skills will be required ten years after graduation.

I do believe there are always trends, and finding those trends can be difficult. Most of the data we gravitate towards, is data that we are directed to look at. The trick to finding trends, is to find new questions to ask. In order to find those questions, I try and look at data through a variety of lenses.

College Admissions Data

The National Association for College Admission Counseling (NACAC) publishes a report called the State of College Admission. I decided to research the 2014 and 2016 reports (data range from 2006-2015) after being very intrigued by a 2007 article titled, Young, Gifted, and Not Getting Into Harvard. 

The author, Michael Winerip, is an interviewer for Harvard applicants. He made a few profound comments. Here is the excerpt that made me take pause:

Of course, evolution is not the same as progress. These kids have an AP history textbook that has been specially created to match the content of the AP test, as well as review books and tutors for those tests. We had no AP textbook; many of our readings came from primary documents, and there was no Princeton Review then. I was never tutored in anything and walked into the SATs without having seen a sample SAT question.

As for my bean sprouts project, as bad it was, I did it alone. I interview kids who describe how their schools provide a statistician to analyze their science project data.

I started to wonder, aside from academics, are university admission processes valuing all the extracurricular work students are doing, and all the stress and time involved in this competitive process. Many extracurricular options involve technology, and require significant investment in time and money.

The data from NACAC was interesting. There are four common summary columns: Considerable Importance, Moderate Importance, Limited Importance, No Importance.

I decided only to review the change of “importance” in the No Importance category. The first three categories are variable. No Importance is not variable, it is absolute, and reflects a definitive negative statement.

The concept is fairly simply. I have 100 points. I weight each of the four categories until I run out of points. In this game, I can decide to declare something a waste of time and effort by using that last category, No Importance.

Here are the results:

Google Sheet View 

This data is troubling. Aside from IB/AP scores, most internal non-academic criteria are losing importance.

Why Has the Value Decreased?

Looking on Reddit and some forums, I found some interviews between admissions officers, students, and parents. Interestingly, I found a comments from these people that resonated with the Harvard interviewer from 2007.

A few things were clear from this small, but powerful, sample:

  • Students working on or in teams need to clearly explain their roles and their contributions; simply being on a team is not enough.
  • Students working without structure, and on original independent projects, are very interesting to the admissions team.
  • Students working inside of a managed program are not really that different from one another.

The value has not decreased, but the supply of students who are doing the same things, and have the same basic profiles, has increased. The demand for those students is lower than the demand for students who are more independent.

The Maker Portfolio

In 2013 MIT introduced a different option for admissions. They called it (and are calling it) The Maker Portfolio.  “In many respects, the Maker Portfolio has been a resounding success. Over the last two years, more than 2000 students have used it to show us the things they make, from surfboards to solar cells, code to cosplay, prosthetics to particle accelerators. We believe the Maker Portfolio has improved our assessment of these applicants and offers us a competitive advantage over our peers who have not developed the processes to identify and evaluate this kind of talent.”~Chris Peterson, Hal Abelson

Since then, a quick Google Search will reveal other universities are aligning with MIT. Washington University, Tufts, Carnegie Mellon, Stanford, California College of the Arts, and more are now offering this option for admissions.

What does this all mean to K-12 education and educational technology?  Activities that used to be hobbies, now need to be student lead within the curriculum. Students need to find an interest, and develop it themselves with as little support as possible. A student should be able to articulate their specific contributions, failures, and growth through a variety of methods, and in a very succinct manner.

Technology investment must shift not only to equipment and resources that allow students to plan and create, but also to systems that will help schools optimize schedules and planning to ensure academic rigor is not sacrificed.

I encourage everyone to review all the resources that went into this post, and I leave you with this comment from Chris Peterson from the MIT Admissions department, “Sometimes students ask me if MIT wants students who are well-rounded. I usually say I don’t care as much if you’re well-rounded or pointy, what I care about is evaluating the space enclosed by the shape.”

Resources

  1. Young, Gifted, and Not Getting Into Harvard
  2. Former Ivy League admissions officer reveals how schools pick students
  3. Hi I’m Nelson Ureña, I am a former admissions officer from Cornell and currently an admissions counselor
  4. 10 College Admissions Secrets: An Inside Look From an Elite College Counselor
  5. 3 Hooks in College Admissions
  6. When Makers Apply to College
  7. I’m an MIT Admissions Officer & longtime FIRST person, AMA
  8. Admissions Revolution As 80 colleges unite to create new application and portfolio platform
  9. 2016 State of College Admission Report from NACAC
  10. 2014 State of College Admission Report from NACAC

 

 

 

Tony DePrato

How Dirty is Your Data?

Leave a comment

Data-Cleaning

By Tony DePrato | Follow me on Twitter @tdeprato

My basic rule for data is, unless there is a life and death scenario unfolding, bad or unclean data is not going to be used. I have yet to encounter a situation where releasing data, which will eventually wreak havoc throughout the school, is an essential and lifesaving endeavor. Delaying systems access due to data issues is difficult. Even the smallest of systems have vocal advocates who will passionately state the damage being done to learning for every day a system is offline.

The best way to exist in a data-driven environment is to be prepared. Being prepared means being aware. Awareness comes from a regular, I would argue monthly, check of all core databases and having policies and procedures for correcting problems.

The real question is this: how does someone not involved in direct data management, check data? And how does someone who is an end user of data set policies to protect the data they need?

Validate and Verify

Anyone dealing with an IT manager, Technology Director, School Information System Specialist, or even a Business Manager should know about validation and verification.

When you validate, you are making certain the contents and format are correct. For example, if I ask you to type your name, and you instead you type your phone number, then you will not be validated. Your data is invalid.

When you verify, you are looking at data that is already in a system to see if it is correct and  in the correct place. For example, when you check a list of student names, you may find all middle names are part of the first name. If this was not by design, you could determine that the data failed verification.

When dealing with assets we often want to verify that what we ordered is what we received. When this type of questioning occurs, verification is happening.

Most people are hit with validation constantly while using the internet. Validation is ubiquitous. Websites often ask you to enter answers to questions, passwords, and CAPTCHA to validate your actions.

These two concepts,validation and verification, are the main tools needed to help the non-data managing people to engage and work with data managers.

Data Auditing

Many people will start auditing data by requesting a spreadsheet of data. This is a mistake.
Data without context is very difficult to understand.

The first step in auditing is simpe. Using questions, learn  how the data is validated:

  1. Where does a new record come from? (Paper + Manual Entry, Online form, Over the Phone + Manual Entry, Software running on computers at the school, etc.)
  2. How are errors prevented and checked?
  3. May I see the…form, paper, call script?
  4. May I do a sample and test the process?
  5. How do I know, after I complete the process, what data was collected?

I often find people are blown away by the time they get to step 5. They are either shocked at how amazing the system is, or appalled at the short comings. Since most data in schools comes from either families or is connected to student assessment, shortcomings do not sit well.

There is nothing to fix at this point. Even if there is a strong belief the system needs to be changed, change should always be data driven, and in this case, driven by the data quality. Until the data is actually reviewed, pause any immediate desire to change things.

The next step is to verify the data, and this can be done by requesting spreadsheets. If the school has a school information system (SIS) like PowerSchool, iSams, Blackbaud, etc. the first set of data needs to come from the SIS. This data should be the primary set used to create accounts in other systems.

Before asking for data, fields must be specified. For example, full name, date of birth, mother’s email address, etc. Be as specific as possible. When people are not specific, data managers take fields and manipulate them. You should be looking at raw data, not data that is filtered and/or edited.

When scanning the data from the SIS, after knowing how the data was collected, errors should start jumping out. If anything seems weird, make a note of it for further discussion. This process usually reveals patterns, such as, everyone having the same zipcode (yes that happened to me).

The secondary systems such as Moodle, Accelerated Reader, Discovery Streaming, etc. can have their data exported to be checked as well. These system often export a .csv file. Don’t worry. Excel, Numbers, and Libre Office can open .csv files. After the file is open, save it as Excel so that it is easier to work with.

Remember, it is not about being certain, it is about being suspicious and asking questions around those suspicions.