Showing posts with label Research Methodology. Show all posts
Showing posts with label Research Methodology. Show all posts

Friday, June 14, 2013

On Methodological approach to Assess Research Output

This a short discussion note, very draft and unedited, on the methodological approach related to the assessment of all research outputs within the centre for public enterprises (CPE), University of the Witwatersrand. I hope this small discussion of mine will add at least modicum of value in an attempt and effort of the centre to improve the quality of research outputs.


Concerning bias intellectually, ideologically or politically


As in the domain of social science, researches on public enterprise are generally qualitative studies, which apply numerous approaches, paradigms, schools and movements encompassed in terms of the ontological, epistemological and methodological assumptions on which they are based. The point is that subjectivity is unavoidable and should not try to avoid it, but needs to be justified in logical and convincing ways.


For instance, there are theoretical and ideological bias relating to the existence or necessity of public enterprises or the desirable size and roles of public sectors in an economy of a nation is natural and unavoidable and should not be afraid of it but just to be aware of it when assessing studies and researches.


Then, a question concerning the role of reviewers arises. 



Who review the reviews of reviewers?


Any academic research article or paper or even book-chapter has to be reviewed by anonymous reviewer or a couple of reviewers before they get published. The due consideration should be given to the matter of choosing reviewers. I would like to suggest that there should be at least two or at best three reviewers, so that their reviews and evaluations should be also assessed and checked against one another. 



Then, what are the characteristics of great journal articles?


To be able write or publish great or at least good articles, the very first step is to know what a great or good article is, what their characteristics and strengths that make them outstanding. According to Litman (2012: 3), “research quality is an epistemological issue (related to the study of knowledge)”. To write or produce good research papers and articles, thorough consideration and conceptualisation, careful preparation and planning, credible and reliable data and information, in-depth and systematic data organisation and analysis are required. However, it also seems to me that there are two important ingredients that make an article or paper make outstanding and significant. These are the question that the research asks and tries to answer or understand, and the impacts that has on the knowledge creation and our understanding about the particular subject. The impact of a research or a article or a paper, in my view, stems from questioning the established conventional beliefs and theories is, in my view, one of the essential characteristics of a great research output.




On questions that are simple but important

In my view, a great research or research journal starts with a simple but important question which questions the conventional beliefs or views or even theories. Most of the new discoveries that revolutionise our thinking and understands are started with a simple but overlooked questions, for example, Galileo’s question that asks “why the Moon orbits around the Earth, while other planets, including the Earth, are orbiting around the Sun” (Millican, 2009). Similarly, Newton asks a simple question: “Why an apple but not the Moon falls down to the ground”.



Logically sound but untrue 


Logical analysis and reasoning are bounded by known facts. In the time when there was no telescope and unable to travel around the world, it looked very logically sound and convincing the structure of Aristotle’s Universe. However, when we have more advanced tools, instruments and technologies, then, in turn, acquired more completed information and data, assumptions and theories that are previously sound and logical become illogical and untenable to answers questions that arised together with new data and information, for example look at two systems of Universes, by Aristotle and by Ptolemy.




Figure 1: Aristotle’s Universe versus Ptolemy’s Universe


  
Aristotle’s Universe
Image from Galileo of Physics department of the University of Virginia,  http://galileo.phys.virginia.edu/classes/109.mf1i.fall03/AristotleToCopernicus_files/image001.gif

Ptolemy’s Universe
Image from From Physics department of the University of Wisconsin (Madison),
http://wisp.physics.wisc.edu/astro104/lecture5/lec5_print.html


Another instance is between Newton and Leibniz on the concept and nature of space, time and motion, and their relationship among each other. While Newton Laws were well-established and accepted, Leibniz argues, against Newton, that space, time and motion are relative, not absolute. In one of his letters to Clarke, Leibniz wrote "As for my own opinion, I have said more than once, that I hold space to be something merely relative, as time is, that I hold it to be an order of coexistences, as time is an order of successions" (Leibniz, 1956: 25-26). Einstein (1905) later proved that Leibniz is damn right in his historic paper that does not have any references to any other publications, although it uses many of the ideas that had already been published by others, however, its substantial contribution is an introducing of a theory of time, distance, mass, and energy.



Portrait of Gottfried Leibniz (1646-1716), German philosopher (oil, circa 
College of Liberal Arts & Social Sciences,  University of Houston



Portrait of Isaac Newton (1642-1727) (Oil, 1689)
Image from Isaac Newton Institute for Mathematical Sciences,



Albert Einstein
From National Public Raido (NPR), 
http://www.npr.org/blogs/13.7/2011/09/28/140839445/is-einstein-wrong


Another memorable example includes a case between “Phillips curve”, and then the concept of NAIRU (Natural Rate of Unemployment). While people were so convinced by and comfortable with the theory of “Phillips curve”, there were two persons Friedman (1968) and Phelps (1967) who dared to question the validity of it, and are also capable to test it against the data from different contexts, and finally repudiate it once and for all. The point of Phillips curve is simple: there is a negative relationship between the unemployment rate and the price level. When the unemployment rate declines, the overall price level (i.e., inflation rate) rises (Phillips, 1958).
Important works are simple but very original and different from the majority of the articles and papers.

Some articles and papers could be very convincing logically and analytically but it does not mean that their data or evidence or results or conclusion is right. There are famous fake scientific and research articles that were first accepted as great and later found out their fakeries (Litman, 2012: 12; Scheer, 1988; Nash, 1996). Litman (2012: 2) calls them “manipulating researches”, which are constructed with, in the terms of Frankfort (2005), bullshit or manipulative misrepresentations). Frankfort argues that bullshit (manipulative misrepresentations) is worse than an actual lie because it denies the value of truth. “A bullshitter’s fakery consists not in misrepresenting a state of affairs but in concealing his own indifference to the truth of what he says. The liar, by contrast, is concerned with the truth, in a perverse sort of fashion: he wants to lead us away from it.”


On the other hand, an article or a paper, which seems to be "dull . . . really dull" and language composition could be lower quality and standard, could also be a seminal work based on its ingenious methodological approach, or its important question and findings, or its impacts on future studies and knowledge creation, for example Keynes’s General Theory, that, according to Paul Samuelson (cited in Moggridge, 1992, cited again in Strathern, 2002: 281) is "badly written, poorly organised .... it is arrogant, bad-tempered, polemical and not overly generous in its acknowledgements. It abounds in mare's nests and confusions". However, Samuelson continues, "in short, it is a work of genius" and Strathern (2002: 281) regards it as the best single-volume biography of Keynes’ life, times and ideas.


It is, according to Nash (1996: 67), because of a fact that research journal or paper writing involves two skills: research and writing. The point is that there are two parts in a research journal or paper their qualities need to be assessed. In my view, theoretical and methodological is more important than the language composition because it can easily improved by language editors. In short, my point is that the prime focus should be on the assessment of research problem formulation and theoretical and methodological choices, rather than of language composition or presentation.



Then, what should we do? 


The first step is to understand what outstanding research papers or articles are and then share these knowledge among us. In order to do them, we have to, need to, prepared and are ready to learn from the masters and stand on the shoulders of giants, as Newton did, and then to imitate them and follow their foot-steps. There is no new thing under the Sun. Best practices and lessons of experiences are readily available if we know how to find them. 

Matters concerning data collection

One of the life-blood of a research and a research journal or paper is data collection. Its quality hinges on the quality of data it used and on the soundness of data-analysis. There are a few matters that I am aware about data-collection. 



Validity of data versus confidentiality and anonymity of respondents


A conflict can be arisen from these standard practices of researching. When respondents demand not to be named and also not to be tape-recorded, the researcher, then, has to rely only on his or her note. Then how can reviewers or examiners validate the authenticity, defensibility, rigour and credibility of the data provided by the researcher? 



Limited usefulness of company’s reports


It is a concern about the reliability of data used in the study. Companies’ reports are necessary to be studied but not reliable; that should be reviewed them cautiously and critically. It is because “companies report what they did, not what they did not do. So finding errors of omission is inherently harder than finding errors of commission” (Carroll & Mui, 2008: 87). Triangulation in terms of data collection is, therefore, indispensible.



Necessity to be aware of differences among cases that are compared


When providing lessons of experiences or best practise, there are some points that need to be aware of and considered carefully. Constraints and challenges of enterprise could be quite similar but they can be different to one another in terms of, for example, types as well as sizes of industries, cultural, social and even religious backgrounds, political backdrops, and institutional structures. 


The same mechanism, tools and processes, as well as capacity, would not always produce the same results in different organisations that have different organisational cultures, and different attitudes of people in them. Simon (1991) also argues that a successful organization is one that musters organizational loyalty among its members, as otherwise the organization would have to spend enormous amounts of time and resources in bargaining and monitoring their performances. There is one interesting and instructive illustration that supports that argument. It is a remark from a top executive of the famous Japanese company Kobe Steel, which is retold by Chang and Singh (1997: 868): The Japanese executive claimed that because of the diversity and complexity of the company's operations, there was no way board members and management could make an informed decision about most of the projects that their staff present to them. However, it is necessary for them to worry and they are not worried because they believe that their company members were mostly doing their best to advance the interest of the company.  


Another consideration needs to be make is about the nature and practices of management of an enterprise. There are quite well-known theories concerning management nature, called Theory X and Theory Y assumptions of manager or management. Performance of an enterprise or an organisation is largely dictated by the nature, practices and quality its management. Managers have their own assumptions about their staff.



Creating an encouraging environment


Another thing what we should try and succeed is to create an environment that are conducive and supportive to researchers and students and, in turn, empower, enable and assist them to be able to produce, write and publish great research papers and articles.


In history, we can see that great thinkers, writers and persons who creates outstanding works tend to come from a team that is comprise with like-minded, so brilliant, people who motivate, inspire, challenge and push to higher level of standard each other. There are plenty of examples of it, such as the Bloomsbury Group grouped some of the finest and brightest authors, philosophers and economists such as John Maynard Keynes, G. E. Moore, Bertrand Russell. E. M. Forster, Lytton Strachey, Virginia Woolf; the Austrian School of economics that is composed by great Austrian economists such as Ludwig von Mises, Friedrich von Hayek; the Chicago school of economics that are a home of Frank Knight, Aaron Director, George Stigler, Milton Friedman, just to name a few of them, where they met together in frequent intense discussions that helped set a group outlook on economic issues, based on price theory, and that has fielded more Nobel Prize laureates and John Bates Clark medalists in economics than any other university or even a country.  



The house of the Bells, Charleston Farmhouse, 
in Bloomsbury where the group used to get together.
Image from Periwork (English online learning resources & News database portal,
http://www.periwork.com/peri_db/wr_db/2006_February_18_10_21_43/part1.html



The Bloomsburg Group
Image from James Madison University,
http://www.jmu.edu/honorsprog/abroad.shtml



It is important to understand that these groups are first thought on, conceptualised, mobilised, organised, founded, nurtured and sustained by a person or a couple of persons who are visionary and forwarded minded. In the case of Bloomsbury Group, it was G. E. Moore; Paul Samuelson and Robert Solow had transformed the MIT (Massachusetts Institute of Technology) to become a great economic department. Similarly, those were Carl Menger Ludwig von Mises, and Friedrich von Wieser who founded and developed the Austrian School of economists.


Conclusion
To recap, to improve research output as well to groom and mentor researchers and students to become outstanding researchers, so that they can produce remarkable and significant research papers and articles, it is important and indispensible to understand what great research papers and articles are and then to build an environment that are conducive, supportive and encouraging, stimulating to have a vibrant and powerful research society or team.


While writing this note, I tried to remember to ask regularly myself a question “who do you think you are to talk and discuss these issues?” This is just sharing what I think important and just means to provoke a brain-storming or more debates. And I believe that at least one point of my discussion in this note will be of some help for at least one person.







References 


Einstein, A. (1905) On the Electrodynamics of Moving Bodies. In Annalen der Physik, 17 (1905), pp. 891-921. (English translations by W. Perrett and G.B. Jeffery from the German Das Relativatsprinzip, 4th ed., published by in 1922 by Tuebner). Retrieved on 6 June 2013 from http://www.pitt.edu/~jdnorton/teaching/HPS_0410/ assignments/04_origins/On-the_electrodynamics/index.html
Frankfort, H. G. (2005) On Bullshit. Princeton: Princeton Press.
Friedman, M. (1968) The Role of Monetary Policy. In American Economic Review, 58 (May 1968), pp. 1-17. Retrieved on 6 June 2013 from www.aeaweb.org/aer/top20/58.1.1-17
Litman, T. (2012) Evaluating Research Quality: Guidelines for Scholarship. Retrieved on 5 June 2013 from www.vtpi.org/resqual.pdf
Moggridge, D. (1992) Maynard Keynes: An Economist’s Biography. London: Routledge.
Nash, T. (1996) The Fake Research Paper: A Creative Approach to Modern Language Association Style. In English Education, Vol. 28, No. 1 (Feb., 1996), pp. 67-71. Retrieved on 6 June 2013 from http://www.jstor.org/stable/40172889
Millican, P. (2009) General Philosophy. A series of lectures delivered by Peter Millican to first-year philosophy students at the University of Oxford. The lectures comprise the 8-week General Philosophy course and were delivered in late 2009, which are watchable and downloable at http://www.philosophy.ox.ac.uk/podcasts/general_philosophy
Phelps, E. (1967) Money Wage Dynamics and Labour Market Equilibrium. In Journal of Political Economy, 75 (July-August 1967), pp. 678-711. Retrieved on 6 June 2013 from http://stevereads.com/papers_to_read/money-wage_dynamics_and_labor_market_equilibrium.pdf
Phillips A. W. (1958) The Relation between Unemployment and the Rate of Change of Money Wage Rates in the United Kingdom, 1861-1957. In Economica, New Series, Vol. 25, No. 100 (Nov., 1958), pp. 283-299. Retrieved on March 22, 2013 from  http://www.jstor.org/stable/2550759
Scheer, S. C. (1988). The fictitious term paper. In Journal of Teaching and Writing, 12, 223- 229. Retrieved on 6 June 2013 from http://journals.iupui.edu/index.php/ teachingwriting/article/view/940/907
Strathern, P. (2002) Dr Strangelove’s Game: A Brief History of Economic Genius. London: Penguin.

Sunday, May 6, 2012

On Researchal Significancy


Have you ever got a thought, at least fortuitously, that one day findings from your research, which is boring, demanding, exacting that you have been doing for ages, all alone, through being broke so often, unnoticed and unimpressed by people around you perhaps those include your spouse or your girl-friend or boy-friend, even your parents, brothers or sisters, or your comrades, or your pastors or monks or boring Pope, or your government, your opposition, will, forever, for once and for all, change the way we understand about our lives or the nature, or improve our living standards exponentially or the way our government perform its task that actually serve our interest, or provide the better ways to treat or cure the diseases? (I know, I know, my sentence is too long. But remember Keynes’ General Theory is also badly organised and difficult to read. In fact, according to one of his greatest admirers, Paul Samuelson, who was also an author of very famous economic text book, Keynes' General Theory is "badly written, poorly organised .... it is arrogant, bad-tempered, polemical and not overly generous in its acknowledgements. It abounds in mare's nests and confusions". However, he continues, "in short, it is a work of genius" (cited in Moggridge, 1992, cited again in Strathern, 2002: 281. I am going to write a lot of posts on Keynes.).

Image from Barnes & Noble,

Trust me, never, ever, forever underestimate the significant of your research that you love dearly, you are so passionate about, you live with it, sleep with it, and have been giving up everything just for it.

It might not happen suddenly, but it will really have invaluable impact in one way or another as long as you believe in yourselves and in what you are researching correctly, professionally, diligently.
With that belief, single-mindedness, and gigantic crazy dream, with a little bit of arroganceness, I stick to my research and my writings, no matter what.

Let’s see one example to support my argument.

It was happened about 80 years ago: to be precise, it was on February 13, 1929.

The place was at St. Mary’s Hospital in London.

It was just a normal research paper reading at the Medical Research Club.

The paper was read by (oh!, no, I will not tell the name of the researcher, but I will do it later.)

The audience at the club was apathetic. No one showed any enthusiasm for the paper.  As Leedy and Ormrod (2001: 43) rightly observe, “great research has frequently been presented to those who are imaginatively both blind and deaf.” (Now, I understand why research grant organizations are not so keen about my research. They are just imaginatively blind and deaf. All of them will be greatly regretful for being uninterested to grant research grants for my research project.)

Although his colleague and audience at the club were indifferent, unimpressed, and apathetic, he knew the value of what he had done, what he had found from his research.  He knew how significant his research outcome really is.

It was actually one of the greatest moments in 20th-century medical research (Leedy & Ormrod, 2001: 43).

Fifteen years later from that day the researcher read his paper to imaginatively blind and deaf audience, he was awarded the Nobel Prize in medicine, together with two other researchers. It was in 1945.

His name is Alexander Fleming, or Dr Alexander Fleming, or Sir Alexander Fleming.

Fleming (centre) receiving the Nobel prize from King Gustaf V of Sweden (right) in 1945.
Image from the Wikimedia Commons http://en.wikipedia.org/wiki/File:Nobelpristagare_Fleming_Midi.jpg

The paper he read at the club was about his research on penicillin. It was in fact presentation of one of the most significant research reports of the early 20th century.


Fleming was named by Time magazine, in 1999, as one of the 100 Most Important People of the 20th Century for his discovery of penicillin, by stating that “it was a discovery that would change the course of history. … the most efficacious life-saving drug in the world, penicillin would alter forever the treatment of bacterial infections” (Time, 29 March 1999).

The procedures of great research (Nobel research, in my term) are exactly the same as those of what we, students, follow in doing our dissertation, thesis, research report. “All research begins with a problem, an observation, a question. Curiosity is the germinal seed.
     Hypotheses are formulated.
     Data are gathered.
     Conclusions are reached” (Leedy & Ormrod, 2001: 44).

(I did try to paraphrase, but the original composition is so beautiful and so perfect. So I gave up. It’s better to quote directly, faithfully :-)


(I am going to tell about Charles Goodyear soon, who lived his whole life, and gave up everything just for a single purpose.)


References

Moggridge, D. (1992) Maynard Keynes: An Economist’s Biography. London: Routledge. According to Strathern (2002), it is the best single-volume biography of Keynes’ life, times and ideas.
Leedy, P. D. and Ormrod, J. E. (2001) Practical Research: Planning and Design (7th Ed). Upper Saddle River (New Jersey): Merrill Prentice Hall.
Strathern, P. (2002) Dr Strangelove’s Game: A Brief History of Economic Genius. London: Penguin.

Note: The previous title was given as “Significance of Research”, but I felt that it was so boring and then tried to be creative and got that new title; it sounds like so Latin. I am also so pleased with my coincency: I again got that new term deriving from ‘coin’.

Thursday, April 26, 2012

Grounded Theory

“Whenever possible, stay away from it”
“It is too complicated, and difficult to handle and manage.”


Yesterday, I attended a PhD short proposal defence (the very first one. In our school we have to do at least three defences in the course of doctoral project. Some students have to do five or six defences.).

It was so hard for the student (he was clearly disappointed and discouraged, although I tried to encourage, motivate and cheer him up by saying that it is supposed to be hard, challenging and so difficult, and he was not alone who is under severe, deadly, attacks at the defence panel. At defence panel, every (at least almost every) student has the same terrible, intellectually humiliating, experience.  It is, in fact, the panle members' job to question, criticise, interrogate and make sure students know what they are going to do in their long research project. But unfortunately and unintentionally they usually also intimidate us. In fact all of panel member as well as professors, lectures are our intellectual and academic parents.

Although most of the defence panel members were kind, as always they are, and considerate, (however, they still did not restrain asking several hard questions and made couple of cruel, but at the same time very helpful comments) one panel member, who is primarily responsible for research methodology, was so tough and a bit (or may be very) aggressive. I was allowed to ask questions there but I decided not to ask any questions.

One of the lessons that I have learnt at the defence is about grounded theory, so well-known among research students but most of us do not really know about it. In his proposal, the student said he is going to use “grounded theory” as his research methodology.

After the student’s 20 minutes presentation, one panel member, who is mainly responsible for the quantitative research methodology and statistical issues, started asking about sampling size of the study briefly, and then he said then your research is not a quantitative research because your sample size is too small (the student said his study is going to use mixed methods, i.e, using qualitative and quantitative methods). After that he asked about “grounded theory”.  He said it is great if you can handle and successfully apply the grounded theory but in his view, grounded theory approach needs to be structured extremely well. The one panel member quickly interrupted: “my best advice to students is “stay away from the grounded theory whenever possible. It is too difficult and most students cannot manage it.”  The another panel member entered the discussion: “Everybody told me not to do grounded theory, but I did not listen when I did my PhD, and it gave me a lot of trouble, stress, hard works along my PhD project, but I eventually completed my PhD with grounded theory. It is not impossible, it is just too complicated and you need to plan and structure your research methodology very well if you are going to use the grounded theory.”

After the defence, I left the room with deep insatiable intellectual curiosity about that monster called "grounded theory". My research does not use grounded theory; I use “case study” methodology, so I do not really need to bother myself and to spend my time with it but I still cannot control myself to study about it. Okay, then, what really is the grounded theory? The following is a few very short notes on it. I hope my notes will be some helpful for students, who are like me, love studying, learning and reading.


Grounded Theory

Anselm L. Strauss, co-founder of Grounded Theory.
Image from Amazon, http://www.amazon.de/Anselm-Strauss-Klassiker-Wissenssoziologie-Str%C3%BCbing/dp/3896695487

Barney G. Glaser, co-founder of the grounded theory, at his 75th birthday with wife Carolyn. Photo: Hans Thulesius. Image from Wikipedia, http://en.wikipedia.org/wiki/File:Glasr75.jpg. (This photo has been released into the public domain by its author, Thulesius.)
First of all, grounded theory is a research method originated, developed, introduced and popularised by Barney G Glaser and Anselm L. Strauss in their book, The Discovery of Grounded Theory. According to Glaser and Strauss (1967: 2). It is a "systematic, qualitative process used to generate a theory that explains, at a broad conceptual level, a process, an action, or interaction about a substantive topic" (Creswell, 2002: 439).

The aim of the research using grounded theory is, according to Yee (2001) “to know what is going on, to look at areas that have either never been studied before or those that are inundated with disparate theories’.

The Discovery of Grounded Theory: Strategies for Qualitative Research by Glaser and Strauss(1967). Image from Amazon, http://www.amazon.com/Barney-Glaser-Anselm-Strauss-Qualitative/dp/B004RPNIU8

In their book, The Discovery of Grounded Theory: Strategies for Qualitative Research (1967), Glaser and Strauss (1967) stated that most research in their time (around 1967) was designed just to verify existing theories, but not to generate new ones. Instead of exploring new areas that were not covered by existing theories, current (around 1967) researchers were just eeking out small pieces of knowledge from existing “grand theories”.

Grounded Theory takes a research approach, which is contrary to most of the more conventional research models (Figure 1). Unlike conventional research methods, in grounded theory approach, data collection, coding and analysis are done immediately, concurrently, and throughout. The process is not impeded by the development of research problems, theoretical understanding or literature review (Jones, Kriflik & Zanko, 2005: 6).


Figure 1. Comparison of Conventional Research Methods to Grounded Theory (Jones, 2005, cited inJones, Kriflik & Zanko, 2005: 7).
  

 I will talk more about it later. I have to do some urgent works now :-)


Reference


Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. New Jersey: Merrill Prentice Hall.
Glaser, B. G. & Strauss, A. L. (1967) The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, Aldine Publishing Company.  
Jones, M. L., Kriflik, G., and Zanko, M. (2005) Grounded Theory: A theoretical and practical application in the Australian Film Industry. Retrieved April 26, 2012, from http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1090&context=commpapers
Yee, B. (2001). Enhancing Security: a Grounded Theory of Chinese Survival in New Zealand. Education Department, University of Canterbury. Retrieved April 26, 2012, from http://ir.canterbury.ac.nz/handle/10092/1771