The Context of Statistics

 

Anurag Behar, CEO of the Azim Premji Foundation, wrote an article yesterday in the Mint, describing how private and government schools are equally incompetent to provide good education. In the article he provides a statistic:

The OECD Programme for International Student Assessment, which places us on learning levels at number 73 in a list of 74 nations, just above Kyrgyzstan, concludes that there is no difference in learning levels across private and government schools. The study that I refer to above concludes that learning is better for children who stayed back at government schools, versus those who were moved to private schools, using financial support on offer as a part of the research design.

(Via Alike in incompetence – Views – livemint.com)

I am assuming the “us” refers to India and we are second last in that list.

I went on to the OECD site, picked up the PISA 2009 Plus Results : Performance of 15-year-olds in reading, mathematics and science for 10 additional participants. [PDF, 193 pages 13MB].

I searched for India on the list. It wasn’t there. Instead, two separate regions were listed under the country name: Himachal Pradesh and Tamil Nadu.

First, it seems obvious to me that two states participating in a survey hardly are representative of a nation. Secondly, it said in the report, that:

Himachal Pradesh-India and Tamil Nadu-India did not meet PISA standards for student sampling. Due to irregularities in the student sample numbers, it was established after the testing that these economies sampled from student lists that were often incomplete: not all 15-year-olds within the school were listed. It was not possible to determine whether any bias existed in the obtained sample. Caution should be exercised when using the data from Himachal Pradesh-India or Tamil Nadu-India and when interpreting the reported analyses.

It is no secret that education in India is facing severe problems, but to take unqualified results from a very small sample and apply it to the nation, is another thing.

The India/China Language Proficiency Debate

Make the Shift

Henry Foy, in the Reuters blog, asks if India has squandered its English advantage?

But, as Asian rival China surpasses India’s English proficiency rates for the first time, that advantage over other developing economies looks to have been squandered.

China was ranked one place above India in Education First’s 2011 English Proficiency Index, released last month, the first time India has been beaten by its neighbour and fellow BRIC economy in the international rankings of foreign countries English-speaking abilities.

(Via Has India squandered its English advantage? | India: A billion aspirations | Analysis & Opinion | Reuters.com)

Mr. Foy uses the EF EPI report (PDF) to ask this question. An interesting question and a topic for debate, given the great English leaps that our eastern neighbour has initiated for the last few years. And an important point for all Indians; except, Henry Foy misses out mentioning a few important factors in his article. However, I do not intend to single out Mr. Foy – other magazines and newspapers have taken the same stand.

For one, while this is an index, there is no attempt at defining proficiency. Here’s what I found on Wikipedia

Language proficiency or linguistic proficiency is the ability of an individual to speak or perform in an acquired language. As theories vary among pedagogues as to what constitutes proficiency, there is little consistency as to how different organizations classify it. Additionally, fluency and language competence are generally recognized as being related, but separate controversial subjects. In predominant frameworks in the United States, proficient speakers demonstrate both accuracy and fluency, and use a variety of discourse strategies. Thus, native speakers of a language can be fluent without being considered proficient. (Via Language proficiency)

But since we do not know the parameter(s) that EF EPI (English Proficiency Index) defined for this test, it anyway doesn’t matter. The report itself seems to be quite unsure of this: “Within the English-teaching community, there is no consensus on the best ways to evaluate English proficiency, or indeed on the ultimate goals of English study. While most English teachers and students agree that communication is the primary objective, more work must be done to define target competencies and how each competency can best be evaluated.“.

The report also mentions the shortcomings of the CEFR (Common European Framework of Reference) for Languages):

However, the CEFR is only a first step towards standard-setting in language education. More detailed definitions of finer-grained skill levels and accompanying evaluative tools are needed, particularly those which take into account current thinking on communication as the primary goal of English study. The most widely-adopted English competency tests today are still heavily weighted towards an older notion of proficiency, no longer in sync with the role that English plays in the world today as an international communication tool.

Secondly, in the Asia EF EPI section, these few lines caught my eye – and I wondered if I was reading it right:

Indeed, although it is very difficult to measure the number of people who speak English in each country because of different definitions of proficiency, the British Council estimated in 2010 that India had anywhere between 55 and 350 million English speakers while a report published by Cambridge University Press estimates that China has 250 to 350 million English learners.

You can see that the report sounds apologetic over the inability to define proficiency. But do read the quote again – if you missed it – India has up to 350million speakers while china has up to 350million learners. Now, the GER for primary and secondary in India has increased from 66% in 1999 to 84% in 2008. Also,

Official statistics on the number of children enrolled in recognised English medium schools in the country show that it has more than doubled within just half a decade from over 61 lakh (6.1million) in 2003 to over 1.5 crore (15million) in 2008.

[…]

In 2006, English as a medium of instruction was fourth — behind Hindi, Bengali and Marathi — but by 2007, it had climbed to second place and grew even further in 2008, beginning to eat into the Hindi numbers.”

(Via August, English, Nagarajan, Rema. The Times Of India. 27 Mar. 2010. Web. 20 Apr. 2011.)

Finally, Mr. Foy’s article fails to mention that the EF EPI test was an online test:

Only countries with a minimum of 400 test takers were included in the index. Countries with fewer than 100 test takers per test on two or more of the tests were also excluded, regardless of the total number of test takers.

We recognize that the test-taking population represented in this index is self-selected and not guaranteed to be representative of the country as a whole. Only those people either wanting to learn English or curious about their English skills will participate in one of these tests. In addition, since the tests are online, people without internet access or unused to online applications are automatically excluded.

Again, not necessarily a good or an accurate sample size. Most teachers or parents in India will not be aware of or have access to this test.

Coming back to the article, while there may be merit in being concerned about losing advantage in a certain skill, I found the ‘straight-line’ interpretation of this report very short-sighted. It is almost as if everyone has looked only at the list – and moved on to the now-fashionable bashing of India’s education problems. While we may have a real problem on hand, this report is no indicator of the problem or the extent of the problem.

Tracking Innovation in Education

George Siemens plans to start a website to track innovation in education around the world. I have signed up to help the ‘trendspotting’ exercise. Here’s the ethos of that website:

Education has an uneasy relationship with businesses. I tried, unsuccessfully, to communicate that I viewed entrepreneurs as risk takers who take ownership of an idea or concept and strive to produce systemic impact. Certainly there is a financial component to the process, but I’m more interested in people generating and testing out new ideas. With full recognition that entrepreneurial activity and education do not share the same ideals and values, I find the need for innovation in education to outweigh this conflict. And I don’t see suitable or viable models for new idea generation and broad implementation outside of entrepreneurship.

Visit the link below for more information and the link to the Google Group, if you’d like to join in!

(Via elearnspace › The urgent need for education/learning tech entrepreneurs)

iNACOL Releases Report on Policy and Funding Frameworks for Online Learning – iNACOL

iNACOL Releases Report on Policy and Funding Frameworks for Online Learning – iNACOL: “The International Association for K-12 Online Learning (iNACOL) announces the release of a new report, Policy and Funding Frameworks for Online Learning. Susan Patrick, President of iNACOL, stated, ‘There is a tremendous need for policy leaders to understand policy and legislative frameworks that enhance and support online learning in their states, to expand access to high-quality online schools and courses for every child. This report decodes policy and practices to help state leaders understand how to be more effective in supporting online and blended school programs that offer more engaging, personalized learning with today’s students.'”

(Via Online Learning Update.)