Hi there, I'm in the midst of starting a nonprofit focused on early
math learning. The mission of the Early Math Initiative is to promote
mathematical understanding and problem solving among young children with
a special emphasis on low-income children ages 2-5 in the United
States. The Early Math Initiative provides high-quality, one-on-one
interactions with trained volunteer 'buddies' in early ed settings such
as preschools or daycares, plus family support and in-home games &
materials, to develop the critical early math skills, confidence, and
joy, which young children need to be successful in school and in life.
Draft Logic Model -
Resources
* Volunteers for 1-on-1 interactions & game nights
* Research-based activities & games
* Connections with low-income daycares and preschools
* Connections with local colleges & businesses to recruit volunteers
* Funding for materials, online resources, & organizational support
* Space to hold family game nights
* Manufacturing capabilities for games & materials
Activities
* High-quality, 1-on-1 interactions between young children & volunteers in early ed settings
* Family support through game nights & take-home games/materials
* Volunteer training in early math best practices
* Online sales of games & materials on a 'one-for-one'/'Toms Shoes' model
* Online videos demonstrating 'math talk' in everyday activities
Outputs
* Number of volunteers trained and interacting
* Number of early ed settings (daycares, classrooms, etc)
* Number of children participating
* Number of parents attending game nights
* Number of games sold online
* Number of online video views and interactions
Outcomes
* Children have higher early math skills at end of program
* Children report enjoying math, feeling competent, and seeing math as useful
* Children enter Kindergarten with higher readiness skills
* Parents engage in more 'math talk' and games at home
* Parents report feeling more positive about math & their own problem solving abilities
* Volunteers report more connection to their community and more awareness of how math is all around us
Impact
*
Children maintain higher math skills, confidence, & enjoyment
through elementary and high school, including higher academic grades
* Children graduate high school at higher rates
* Children attend college at higher rates
* Children have better career and financial success in adulthood
1)
What are the key lessons and insights you have about your
organization’s financial resources as a result of developing your draft
logic model?
One of the insights I got through doing the logic model
was actually more about human resources - many of the resources we need
to do the activities can be achieved through a combination of finances
and human capital. Certain things have to be 'bought', but other
aspects could be found or 'traded' through connections.
Looking
through the outcomes and the impact also reminded me to focus resources
(financial and otherwise) on the activities that do show the results
that we're looking for.
2) Did any parts of your logic model surprise you? Which ones and in what ways?
Working
through the logic model short-term outcomes surprised me to think about
what else are we hoping to see besides just raising test scores, and
then how will we measure those outcomes (how do you measure enjoyment of
math?). Also, the children are important, but we want to see change
for the parents/families and for the volunteers as well.
3) Are
there any action steps concerning your organization’s financial
resources as a result of developing your draft logic model? What do you
hope to accomplish by pursuing these action steps?
At this point in
the start of the nonprofit, we're mostly looking for funding, so
clarifying our logic model is wonderful to be able to better articulate
to potential funders what impact we're going to have. I will start
including the full model in larger fundraising, and mentioning the key
aspects in smaller fundraising.
Tuesday, September 30, 2014
Wednesday, September 24, 2014
Week 3 Post for Educational Policy - Desegregation
Prof Peterson describes the Brown decision as "the signal accomplishment of the 20th century as it transformed opportunity for Black Americans and created the opportunity to create a more egalitarian society consistent with the nation's ideals" (Desegregation III, 9:00mins). Although that decision resulted in both primary and secondary accomplishments for Black Americans, the benefits were not as drastic and immediate as many had hoped at the time. Today, the U.S. still cannot claim racial equality in student achievement and attainment outcomes.
Please describe some of the primary and secondary accomplishments that are associated with the Brown decision. Further, what policies or constitutional changes should we now pursue to realize the goals of racial equality in educational outcomes? Make sure to acknowledge potential implementation challenges that might accompany your proposed policies (such as the interracial violence that erupted in response to forced busing in the 1970s).
Example 1: TA Anna evaluated the effectiveness of a statewide voucher program in its effectiveness at reducing racial stratification in Louisiana's public schools. What's your evaluation of this proposal to use school vouchers as a tool to better integrate both public and private schools? Is that goal worthy in and of itself or should we be more focused on improving test scores of minority students? http://educationnext.org/the-louisiana-scholarship-program/
Example 2: TA Anna also evaluated the effectiveness of teacher-student race matching as a strategy to improve minority students' achievement outcomes. She finds positive, statistically significant achievement effects when Black and White students are assigned to race-congruent teachers in reading and for Black, White, and Asian students in math. What's your evaluation of this proposal as a way to close persistent achievement gaps? http://www.uaedreform.org/representation-in-the-classroom-the-effect-of-own-raceethnicity-teacher-assignment-on-student-achievement/
As Prof Peterson describes in the videos, the Brown decision that 'separate but equal' was unconstitutional was definitely a turning point. Not everyone agreed with the ruling and some schools were closed down rather than be desegregated. But on the whole, the Brown decision fueled the civil rights movement, and the Civil Rights Act of 1964 also helped to move the desegregation forward. However, interracial violence was still prevalent, especially around 1968 with the Watts Riots and the assasinations of Martin Luther King Jr and Robert Kennedy. In the Milliken versus Bradley case in 1974, the courts decided that de facto segregation was ok, as long as de jure (or explicit law) segregation wasn't happening. This distinction has led within district segregation to go down over the years, but between district segregation to go up. Within districts, black and white students attend the same schools and aren't segregated, but black and white families often choose different places to live, and thus the school districts are often segregated in black districts and white districts.
While the Brown decision has had a huge impact, we still have significant racial inequality in educational achievement. Part of this may be because of the between district segregation - but how do we integrate different ethnicities? In the past, bussing has been tried - but people often prefer to go to schools closer to where they live, and is it really the best use of the children's time to have them in transit for hours each day? Perhaps giving families a choice of schools would be the best option? But as the All Over the Map reading (http://educationnext.org/all-over-the-map/) brings up, poor families often don't have the ability or the information to adequately evaluate schools in order to make the best educational choice - or they may value other aspects besides high quality instruction, such as proximity to caregivers or maintaining previous school community connections. And even if schools start to become more integrated, there are often major cultural clashes as the Elephant in the Classroom reading (http://educationnext.org/the-elephant-in-the-classroom/) points out. Different ethnic groups and social classes sometimes have different ideas of what school should be like, and how teaching should happen. I've done some reading about KIPP schools, which can seem very strict to educators more accustomed to progressive teaching methods - but after the Elephant in the Classroom article, I have a better sense of how KIPP's culture fits in with the culture of the students it's trying to serve.
But what do we need to happen in order to achieve racial equity in education? The Lousiana Scholarship program shows that at least in some situations, school choice can result in better integration. While I would argue that integration is a worthy goal for helping students to have a broader perspective and to open up their filter bubble (the LSP article made me more aware of the filter bubble I live in - I had no idea that the government was still monitoring schools for compliance with desegregation laws), I would want to see data on whether the integration actually improves educational outcomes. If integration means more students attending high-quality schools, then my guess is that there would be higher educational equity, but if integration just means moving students around within low quality schools, then even if there's more equity between ethnicities, it wouldn't work if that equity point is low. The goal is not just to be equal - it's to have everyone achieve higher.
The Representation in the Classroom paper in some sense argues against integration, because if students learn better in classrooms where their teacher is the same race or ethnicity, then maybe we should segregate all students into black classrooms and white classrooms with the appropriate teacher? If this would actually result in consistently higher academic achievement, then perhaps it is an option to be made available to families, in a similar way that some schools are all-girls or all-boys. Although I do wonder if only learning one cultural norm may have negative future effects. Students especially in this day and age need to know how to deal with all different types of people in order to move forward professionally. Perhaps if we had high-quality teachers in integrated classrooms who had high expectations for all of their students, no matter what race or ethnicity...
Please describe some of the primary and secondary accomplishments that are associated with the Brown decision. Further, what policies or constitutional changes should we now pursue to realize the goals of racial equality in educational outcomes? Make sure to acknowledge potential implementation challenges that might accompany your proposed policies (such as the interracial violence that erupted in response to forced busing in the 1970s).
Example 1: TA Anna evaluated the effectiveness of a statewide voucher program in its effectiveness at reducing racial stratification in Louisiana's public schools. What's your evaluation of this proposal to use school vouchers as a tool to better integrate both public and private schools? Is that goal worthy in and of itself or should we be more focused on improving test scores of minority students? http://educationnext.org/the-louisiana-scholarship-program/
Example 2: TA Anna also evaluated the effectiveness of teacher-student race matching as a strategy to improve minority students' achievement outcomes. She finds positive, statistically significant achievement effects when Black and White students are assigned to race-congruent teachers in reading and for Black, White, and Asian students in math. What's your evaluation of this proposal as a way to close persistent achievement gaps? http://www.uaedreform.org/representation-in-the-classroom-the-effect-of-own-raceethnicity-teacher-assignment-on-student-achievement/
As Prof Peterson describes in the videos, the Brown decision that 'separate but equal' was unconstitutional was definitely a turning point. Not everyone agreed with the ruling and some schools were closed down rather than be desegregated. But on the whole, the Brown decision fueled the civil rights movement, and the Civil Rights Act of 1964 also helped to move the desegregation forward. However, interracial violence was still prevalent, especially around 1968 with the Watts Riots and the assasinations of Martin Luther King Jr and Robert Kennedy. In the Milliken versus Bradley case in 1974, the courts decided that de facto segregation was ok, as long as de jure (or explicit law) segregation wasn't happening. This distinction has led within district segregation to go down over the years, but between district segregation to go up. Within districts, black and white students attend the same schools and aren't segregated, but black and white families often choose different places to live, and thus the school districts are often segregated in black districts and white districts.
While the Brown decision has had a huge impact, we still have significant racial inequality in educational achievement. Part of this may be because of the between district segregation - but how do we integrate different ethnicities? In the past, bussing has been tried - but people often prefer to go to schools closer to where they live, and is it really the best use of the children's time to have them in transit for hours each day? Perhaps giving families a choice of schools would be the best option? But as the All Over the Map reading (http://educationnext.org/all-over-the-map/) brings up, poor families often don't have the ability or the information to adequately evaluate schools in order to make the best educational choice - or they may value other aspects besides high quality instruction, such as proximity to caregivers or maintaining previous school community connections. And even if schools start to become more integrated, there are often major cultural clashes as the Elephant in the Classroom reading (http://educationnext.org/the-elephant-in-the-classroom/) points out. Different ethnic groups and social classes sometimes have different ideas of what school should be like, and how teaching should happen. I've done some reading about KIPP schools, which can seem very strict to educators more accustomed to progressive teaching methods - but after the Elephant in the Classroom article, I have a better sense of how KIPP's culture fits in with the culture of the students it's trying to serve.
But what do we need to happen in order to achieve racial equity in education? The Lousiana Scholarship program shows that at least in some situations, school choice can result in better integration. While I would argue that integration is a worthy goal for helping students to have a broader perspective and to open up their filter bubble (the LSP article made me more aware of the filter bubble I live in - I had no idea that the government was still monitoring schools for compliance with desegregation laws), I would want to see data on whether the integration actually improves educational outcomes. If integration means more students attending high-quality schools, then my guess is that there would be higher educational equity, but if integration just means moving students around within low quality schools, then even if there's more equity between ethnicities, it wouldn't work if that equity point is low. The goal is not just to be equal - it's to have everyone achieve higher.
The Representation in the Classroom paper in some sense argues against integration, because if students learn better in classrooms where their teacher is the same race or ethnicity, then maybe we should segregate all students into black classrooms and white classrooms with the appropriate teacher? If this would actually result in consistently higher academic achievement, then perhaps it is an option to be made available to families, in a similar way that some schools are all-girls or all-boys. Although I do wonder if only learning one cultural norm may have negative future effects. Students especially in this day and age need to know how to deal with all different types of people in order to move forward professionally. Perhaps if we had high-quality teachers in integrated classrooms who had high expectations for all of their students, no matter what race or ethnicity...
Sunday, September 21, 2014
Week 2 Post for Saving Schools
Based on the lecture videos and required readings for this week, discuss what you feel the role of the local school board should be in the 21st century.
The Lost at Sea reading does bring up some good points about who might be on school boards, and the various agendas they would have. This reading as well as the video also talks about how while school boards are elected, they're elected by such a small minority of people that it's not really participatory democracy at work. Like any system or set of organizations, some of them are going to work extremely well, and some not as well. To say that we should do away with school boards because some of them aren't very good is not a solution.
I think we still do need some local accountabilty and a system of checks and balances. Having a school board that can look over the schools and provide some outside perspective is important. As the Steering a True Course reading notes, some school boards are pushing for educational reform and establishing even more rigorous accountability measures than the state. And the same reading talks about how there still need to be financial oversight.
I see the role of local school boards as providing more accountability and oversight, and giving the local community a way to participate in the schools. Now that we have more professional administration (superintendents, etc), the school board doesn't need to be involved in the running of the schools, but rather should be more a board of advisors who can look at the overall picture and help with strategy. We don't need people who are just going to say yes to various measures that the administrators are already doing, we need people who can help the administrators plan and change in order to better serve the students.
The Lost at Sea reading does bring up some good points about who might be on school boards, and the various agendas they would have. This reading as well as the video also talks about how while school boards are elected, they're elected by such a small minority of people that it's not really participatory democracy at work. Like any system or set of organizations, some of them are going to work extremely well, and some not as well. To say that we should do away with school boards because some of them aren't very good is not a solution.
I think we still do need some local accountabilty and a system of checks and balances. Having a school board that can look over the schools and provide some outside perspective is important. As the Steering a True Course reading notes, some school boards are pushing for educational reform and establishing even more rigorous accountability measures than the state. And the same reading talks about how there still need to be financial oversight.
I see the role of local school boards as providing more accountability and oversight, and giving the local community a way to participate in the schools. Now that we have more professional administration (superintendents, etc), the school board doesn't need to be involved in the running of the schools, but rather should be more a board of advisors who can look at the overall picture and help with strategy. We don't need people who are just going to say yes to various measures that the administrators are already doing, we need people who can help the administrators plan and change in order to better serve the students.
Saturday, September 20, 2014
Assignment 2 for Social Entrepreneurship - Logic Model, Beneficiary Experience Table, and Deliverables Table
I am working on a social enterprise supporting early math learning for
preschool children in the US. I've attached the logic model for our
initiative.
Our Beneficiary Experience Table... from the perspective of family of the preschool child
- enroll child in preschool that is participating in the program
- hear about the family math nights from the preschool and our program
- arrange to have the evening available
- find transportation to the school
- attend the family math night
- be instructed in how to play the games
- bring the early math games home
- find time to spend with child
- play early math games during that parent-child time
Our Beneficiary Experience Table... from the perspective of the preschool where the child is enrolled.
- hear about the early math initiative
- connect with our program and arrange to be a participating school
- create time in the school day for the volunteers to come in and play games with the children
- find a suitable time and location for the family game nights
- coordinate with our program for the family game night logistics
- support the volunteers in interacting with the families during game night
- maintain contact with the families to support continued use of the early math games at home
Our Deliverables Table...
- Collect the research about which games are most effective
- Create the curriculum and actual game materials
- Market and use social media to reach out to preschools and families to participate
- Recruit volunteers to go to the preschools and to do family game nights
- Train the volunteers in using the research-backed games
- Coordinate the logistics of the volunteers going to the preschools
- Coordinate the logistics of the family game nights
- Follow up with preschools and families to encourage continued game playing
- Create online videos of best practices in early math activities
Our Beneficiary Experience Table... from the perspective of family of the preschool child
- enroll child in preschool that is participating in the program
- hear about the family math nights from the preschool and our program
- arrange to have the evening available
- find transportation to the school
- attend the family math night
- be instructed in how to play the games
- bring the early math games home
- find time to spend with child
- play early math games during that parent-child time
Our Beneficiary Experience Table... from the perspective of the preschool where the child is enrolled.
- hear about the early math initiative
- connect with our program and arrange to be a participating school
- create time in the school day for the volunteers to come in and play games with the children
- find a suitable time and location for the family game nights
- coordinate with our program for the family game night logistics
- support the volunteers in interacting with the families during game night
- maintain contact with the families to support continued use of the early math games at home
Our Deliverables Table...
- Collect the research about which games are most effective
- Create the curriculum and actual game materials
- Market and use social media to reach out to preschools and families to participate
- Recruit volunteers to go to the preschools and to do family game nights
- Train the volunteers in using the research-backed games
- Coordinate the logistics of the volunteers going to the preschools
- Coordinate the logistics of the family game nights
- Follow up with preschools and families to encourage continued game playing
- Create online videos of best practices in early math activities
Project Proposal for Coursera Data Analysis
In the Data Analysis Coursera class, we get to do a project based on a research question and data that we choose! So even though it's a math/there's only one correct answer type of class, I can post some of my work since it's about the data that I'm choosing rather than everyone using the same data. (I still want to write a post about what I can and can't post here because of whether or not the homework has 'one' correct answer.)
RESEARCH QUESTION: In one sentence, what is your research question?
Is there a relationship between educational achievement, as measured by highest degree, and hours of TV watched per week?
DATA - Citation: Include a citation for your data, and if your data set is online, provide a link to the source
I will be using the General Social Survey dataset as provided for this class - I'm going to be using only the data from after 2000 in order to try to make the results more generalizable to current day.
http://bit.ly/dasi_gss_data
DATA - Collection: Describe how the data were collected.
The data for the General Social Survey has been collected through interviews every year or two since 1972. From the intro pdf at http://publicdata.norc.org/GSS/DOCUMENTS/BOOK/GSS_Codebook_intro.pdf, "Each survey from 1972 to 2004 was an independently drawn sample of English-speaking persons 18 years of age or over, living in non-institutional arrangements within the United States. Starting in 2006 Spanish-speakers were added to the target population." The exact sampling procedure has varied over the years as described in http://publicdata.norc.org:41000/gss/documents//BOOK/GSS_Codebook_AppendixA.pdf
DATA - Cases (observational/experimental units): What are the cases? (Remember: case = units of observation or units of experiment)
The cases are individual adults - English-speaking, 18 and over, not living in institutions; with Spanish-speaking adults added for surveys done 2006 and later.
DATA - Variables: What are the two variables you will be studying? State the type of each variable.
I will be looking at...
1) educational achievement as defined by highest degree - DEGREE. In the GSS data, it's an ordinal/categorical with five levels (Left HS, HS, Junior College, Bachelors, Graduate), so I may break it down into two or three categories, such as High School vs College, depending on the statistical methods we learn and if I can compare that many levels.
2) hours of TV watched a week - TVHOURS. It's numeric with values from 0-24.
DATA - Type of study: What is the type of study? Is it an observational study or an experiment? Explain how you've arrived at your conclusion using information on the sampling and/or experimental design.
The GSS Study is an observational study - the interviewers 'observe' or take note of the participant's answers. There's no treatment or control groups, so it's not an experiment.
DATA - Scope of inference - generalizability: Identify the population of interest, and whether the findings from this analysis can be generalized to that population, or, if not, a subsection of that population. Explain why or why not. Also discuss any potential sources of bias that might prevent generalizability.
The GSS Study hopes to represent the whole US population - or at least English (and now Spanish) speaking adults. Because it's an observational study, it has external validity and can be generalized to the population of interest. The study tries to get a representative sample through its surveying method, although the method has changed over the years of the survey. One thing to note is that the survey answers collected from many years ago may not generalize to the current population (although that data would probably accurately reflect the population of that time), hence why I am only using the data from after 2000.
DATA - Scope of inference - causality: Can these data be used to establish causal links between the variables of interest? Explain why or why not.
No, these data cannot be used to establish any causal links between the variables. Because it's an observational study, and not an experiment, we can only tell if the two variables are correlated - and correlation does not equal causation! In this particular case of education and tv watched, I'm not even sure if causality is really something to think about. The educational attainment came first, so therefore does going to school longer cause someone to watch less TV? Or is it more likely there's a confounding variable of motivation or intelligence or something else that causes both higher educational attainment and less TV watching?
EXPLORATORY DATA ANALYSIS:
Perform a brief exploratory data analysis - just one or two relevant descriptive statistics and visualizations of the data. Also address what the exploratory data analysis suggests about your research question.
Using the GSS data from after the year 2000, I found that the mean hours of TV watched was very different for the different levels of educational attainment. The overall average hours was 2.983, but the average for the different education varied from 1.939 to 4.070, indicating that there is a relationship between education and tv watched.
I also did a boxplot to see the distributions over the different educational levels (attached as a pdf). The distributions seem to be fairly similar, with long trailing tails of outliers going toward the maximum of 24 hours per week. The boxplot shows that the median for left high school and high school is the same (3 hours), and the median for junior college, bachelors, and graduate is the same (2 hours), so perhaps the greatest difference is between the two broader categories of 'high school or lower' and 'any college education'
.
jklj;
RESEARCH QUESTION: In one sentence, what is your research question?
Is there a relationship between educational achievement, as measured by highest degree, and hours of TV watched per week?
DATA - Citation: Include a citation for your data, and if your data set is online, provide a link to the source
I will be using the General Social Survey dataset as provided for this class - I'm going to be using only the data from after 2000 in order to try to make the results more generalizable to current day.
http://bit.ly/dasi_gss_data
DATA - Collection: Describe how the data were collected.
The data for the General Social Survey has been collected through interviews every year or two since 1972. From the intro pdf at http://publicdata.norc.org/GSS/DOCUMENTS/BOOK/GSS_Codebook_intro.pdf, "Each survey from 1972 to 2004 was an independently drawn sample of English-speaking persons 18 years of age or over, living in non-institutional arrangements within the United States. Starting in 2006 Spanish-speakers were added to the target population." The exact sampling procedure has varied over the years as described in http://publicdata.norc.org:41000/gss/documents//BOOK/GSS_Codebook_AppendixA.pdf
DATA - Cases (observational/experimental units): What are the cases? (Remember: case = units of observation or units of experiment)
The cases are individual adults - English-speaking, 18 and over, not living in institutions; with Spanish-speaking adults added for surveys done 2006 and later.
DATA - Variables: What are the two variables you will be studying? State the type of each variable.
I will be looking at...
1) educational achievement as defined by highest degree - DEGREE. In the GSS data, it's an ordinal/categorical with five levels (Left HS, HS, Junior College, Bachelors, Graduate), so I may break it down into two or three categories, such as High School vs College, depending on the statistical methods we learn and if I can compare that many levels.
2) hours of TV watched a week - TVHOURS. It's numeric with values from 0-24.
DATA - Type of study: What is the type of study? Is it an observational study or an experiment? Explain how you've arrived at your conclusion using information on the sampling and/or experimental design.
The GSS Study is an observational study - the interviewers 'observe' or take note of the participant's answers. There's no treatment or control groups, so it's not an experiment.
DATA - Scope of inference - generalizability: Identify the population of interest, and whether the findings from this analysis can be generalized to that population, or, if not, a subsection of that population. Explain why or why not. Also discuss any potential sources of bias that might prevent generalizability.
The GSS Study hopes to represent the whole US population - or at least English (and now Spanish) speaking adults. Because it's an observational study, it has external validity and can be generalized to the population of interest. The study tries to get a representative sample through its surveying method, although the method has changed over the years of the survey. One thing to note is that the survey answers collected from many years ago may not generalize to the current population (although that data would probably accurately reflect the population of that time), hence why I am only using the data from after 2000.
DATA - Scope of inference - causality: Can these data be used to establish causal links between the variables of interest? Explain why or why not.
No, these data cannot be used to establish any causal links between the variables. Because it's an observational study, and not an experiment, we can only tell if the two variables are correlated - and correlation does not equal causation! In this particular case of education and tv watched, I'm not even sure if causality is really something to think about. The educational attainment came first, so therefore does going to school longer cause someone to watch less TV? Or is it more likely there's a confounding variable of motivation or intelligence or something else that causes both higher educational attainment and less TV watching?
EXPLORATORY DATA ANALYSIS:
Perform a brief exploratory data analysis - just one or two relevant descriptive statistics and visualizations of the data. Also address what the exploratory data analysis suggests about your research question.
Using the GSS data from after the year 2000, I found that the mean hours of TV watched was very different for the different levels of educational attainment. The overall average hours was 2.983, but the average for the different education varied from 1.939 to 4.070, indicating that there is a relationship between education and tv watched.
I also did a boxplot to see the distributions over the different educational levels (attached as a pdf). The distributions seem to be fairly similar, with long trailing tails of outliers going toward the maximum of 24 hours per week. The boxplot shows that the median for left high school and high school is the same (3 hours), and the median for junior college, bachelors, and graduate is the same (2 hours), so perhaps the greatest difference is between the two broader categories of 'high school or lower' and 'any college education'
.
by(gssafter2000$tvhours, gssafter2000$degree, mean, na.rm=TRUE) gssafter2000$degree: Lt High School [1] 4.069703 ---------------------------------------------------------------------------- gssafter2000$degree: High School [1] 3.169679 ---------------------------------------------------------------------------- gssafter2000$degree: Junior College [1] 2.641186 ---------------------------------------------------------------------------- gssafter2000$degree: Bachelor [1] 2.230769 ---------------------------------------------------------------------------- gssafter2000$degree: Graduate [1] 1.939353
jklj;
Wednesday, September 17, 2014
Post for Ed Policy - Methodological Thoughts
One of the most policy relevant charter school studies is an
observational study conducted by the Center for Research on Education
Outcomes (CREDO) at Stanford University. This study has high external
validity. (Information on their methodology can be found at http://credo.stanford.edu/documents/NCSS2013_Technical%20Appendix.pdf, pages 5-11, which also discusses the internal vs. external validity concerns discussed above. The main report (http://credo.stanford.edu/documents/NCSS%202013%20Final%20Draft.pdf) also discusses the methodology on pages 8-11, but in less detail.)
CREDO matches each charter school student to public school students from “Charter Feeder Schools” that have similar baseline test scores and the same race, gender, special education status, limited English proficiency status and free lunch status. They then compare how the charter school students perform on low-stakes tests relative to public school students.
Overall, I was very impressed with the amount of data they were able to collect and process, although when analyzing that many different sections and doing so many statistical analyses, they're bound to find some significant results. They do seem aware of this, mentioning it in the issues associated with repeated tests - although also mentioning that they chose not to reduce the significance level, even with so many tests. Given this, we have to look a bit more carefully what is 'significant', especially since so many of the results they found were within .01 standard deviations, at least when aggregating all of the data nationally. Also when matching the VCRs, they did allow variation in the TPS students' starting scores of plus or minus 0.1 SDs - which is often more than the level of difference they're often finding. They do talk that not having exactly equal starting scores, and seem to feel like it's not an issue once averaged out - but to a less statistically trained eye, I wonder.
And while talking about the VCR matching, I'm also wondering if the charter school students are going to show more variation than the VCRs, because each charter student is matched with an averaged VCR including up to 7 TPS students. The score of one is going to show more variance than an averaged score. Again the study does mention this criticism, as put forth by Hoxby, but says that they did more statistical analysis to find that it wasn't a valid criticism. Another interesting point to me was that the matched TPS students had a lower than average change, meaning that the virtual twins did not progress as much as an average student.
On the issue of confounding variables - Because this study wasn't looking at lottery 'winners' and 'losers' who had all tried to get into the charter schools, the main confounder that stuck out to me was around parent involvement and school choice. The charter school students were matched to TPS students on the basis of race, gender, free or reduced price lunch status, special education status, and English language learner status. But what about the parents? The VCRs came from feeder schools where presumably all of the parents knew about the charter schools, and yet only the parents of the charter school students actually enrolled them in a charter school. The analysis states that the VCR method produces results that are not significantly different from experimental methods, and it also points out some issues with using lottery as an experiment (not totally random, not enough students, etc), but I still felt like the difference between who chooses to attend charter schools versus who doesn't would be a large confounding variable. Not sure which way it would affect the results though. Perhaps parents who are more involved would choose charter schools, and their children might do better because their parents are involved and supportive of their education? Or perhaps parents whose children aren't doing well in TPS are more likely to move to a charter school? (The study did mention some evidence of this.) Or perhaps parents who have a higher education level are more likely to research and enroll their children in a charter school, and parental education level tends to correlate with student achievement?
Other possible confounders... Do charter schools attract better (or worse, or newer, or...) teachers? Perhaps any effects are from differences in teacher quality. Do charter schools have different class sizes, or school sizes? I think the study mentioned that charter schools were often smaller schools than the TPS (although I can't find the reference now that I'm looking again). Do charter schools have more instructional time? Some charter schools have longer school days or longer school years - maybe those '7 extra days in reading' results actually came from having seven extra days of instructional time. Do charter schools have a stronger school community? By choosing to go to a school, students might be more engaged with the school community, which may affect math & reading scores. There are a lot of potential differences between charter school students and TPS students that were not accounted for in the VCR method. Thus the results probably cannot be externally valid for all students.
Another big issue that stood out for me was the wide variation in the state scores - and most likely in individual school scores. Charter schools are not a homogenous group, and so to make judgements based on aggregate might not be very helpful. Just because charter schools as a whole are showing basically the same (or possibly a little bit better) results than TPS, tells you nothing about the charter school down the street from you, or even the charter schools in your state. The study does delve a bit into the state differences, but I would have liked to have seen more. I care less about the overall state of charter schools and more about which ones are working and which ones aren't.
CREDO matches each charter school student to public school students from “Charter Feeder Schools” that have similar baseline test scores and the same race, gender, special education status, limited English proficiency status and free lunch status. They then compare how the charter school students perform on low-stakes tests relative to public school students.
Overall, I was very impressed with the amount of data they were able to collect and process, although when analyzing that many different sections and doing so many statistical analyses, they're bound to find some significant results. They do seem aware of this, mentioning it in the issues associated with repeated tests - although also mentioning that they chose not to reduce the significance level, even with so many tests. Given this, we have to look a bit more carefully what is 'significant', especially since so many of the results they found were within .01 standard deviations, at least when aggregating all of the data nationally. Also when matching the VCRs, they did allow variation in the TPS students' starting scores of plus or minus 0.1 SDs - which is often more than the level of difference they're often finding. They do talk that not having exactly equal starting scores, and seem to feel like it's not an issue once averaged out - but to a less statistically trained eye, I wonder.
And while talking about the VCR matching, I'm also wondering if the charter school students are going to show more variation than the VCRs, because each charter student is matched with an averaged VCR including up to 7 TPS students. The score of one is going to show more variance than an averaged score. Again the study does mention this criticism, as put forth by Hoxby, but says that they did more statistical analysis to find that it wasn't a valid criticism. Another interesting point to me was that the matched TPS students had a lower than average change, meaning that the virtual twins did not progress as much as an average student.
On the issue of confounding variables - Because this study wasn't looking at lottery 'winners' and 'losers' who had all tried to get into the charter schools, the main confounder that stuck out to me was around parent involvement and school choice. The charter school students were matched to TPS students on the basis of race, gender, free or reduced price lunch status, special education status, and English language learner status. But what about the parents? The VCRs came from feeder schools where presumably all of the parents knew about the charter schools, and yet only the parents of the charter school students actually enrolled them in a charter school. The analysis states that the VCR method produces results that are not significantly different from experimental methods, and it also points out some issues with using lottery as an experiment (not totally random, not enough students, etc), but I still felt like the difference between who chooses to attend charter schools versus who doesn't would be a large confounding variable. Not sure which way it would affect the results though. Perhaps parents who are more involved would choose charter schools, and their children might do better because their parents are involved and supportive of their education? Or perhaps parents whose children aren't doing well in TPS are more likely to move to a charter school? (The study did mention some evidence of this.) Or perhaps parents who have a higher education level are more likely to research and enroll their children in a charter school, and parental education level tends to correlate with student achievement?
Other possible confounders... Do charter schools attract better (or worse, or newer, or...) teachers? Perhaps any effects are from differences in teacher quality. Do charter schools have different class sizes, or school sizes? I think the study mentioned that charter schools were often smaller schools than the TPS (although I can't find the reference now that I'm looking again). Do charter schools have more instructional time? Some charter schools have longer school days or longer school years - maybe those '7 extra days in reading' results actually came from having seven extra days of instructional time. Do charter schools have a stronger school community? By choosing to go to a school, students might be more engaged with the school community, which may affect math & reading scores. There are a lot of potential differences between charter school students and TPS students that were not accounted for in the VCR method. Thus the results probably cannot be externally valid for all students.
Another big issue that stood out for me was the wide variation in the state scores - and most likely in individual school scores. Charter schools are not a homogenous group, and so to make judgements based on aggregate might not be very helpful. Just because charter schools as a whole are showing basically the same (or possibly a little bit better) results than TPS, tells you nothing about the charter school down the street from you, or even the charter schools in your state. The study does delve a bit into the state differences, but I would have liked to have seen more. I care less about the overall state of charter schools and more about which ones are working and which ones aren't.
Tuesday, September 16, 2014
Intro for Immunity to Change
Hi there, my name is Teresa Gonczy, and I'm an entrepreneur and educator. Although currently I'm taking some time off to get a masters degree in education. Because all of the classes I'm taking are online, I'm actually traveling a lot this year - mostly between Los Angeles, San Francisco, Chicago, and Boston.
By taking this course, I'm hoping to gain some insight into how I'm holding myself back, and then be able to change and move forward. I'm not sure yet what goal I want to work on - I'm debating between some more personal goals, such as finding a spouse and starting a family, or more professional goals, such as leading larger organizations and going from having a local impact to a broader impact.
Looking forward to changing along with everyone over the next weeks! :-)
By taking this course, I'm hoping to gain some insight into how I'm holding myself back, and then be able to change and move forward. I'm not sure yet what goal I want to work on - I'm debating between some more personal goals, such as finding a spouse and starting a family, or more professional goals, such as leading larger organizations and going from having a local impact to a broader impact.
Looking forward to changing along with everyone over the next weeks! :-)
Subscribe to:
Posts (Atom)