Author: Sanjay Goel, http://in.linkedin.com/in/sgoel
In February, 2013, I started taking the classes of Research Methodology for PhD scholars in CSE and ECE departments. In total, 27 scholars registered in this course. All of them had already completed their MTech. recently or some years ago. Most of them were registered as part time candidates and were working in academia/industry. A good number of them were registered as full time candidates and also had prior work experience. They all had already spend 1-3 semesters in their doctoral program and except two all had already completed he required compulsory 3 credit course on literature survey that required them critique and review a minimum of 15 research papers in their area. They were pursuing their PhD in highly diversified areas covering a huge spectrum of electronics, communication, and computing, e.g., microelectronics, wireless communications, VLSI, data mining, software engineering, embedded systems, sensor networks, image processing, information security, NLP, usability, etc. : Hence, the student population had great diversity of research area, age group, work experience, etc.
It was the first time I was asked to teach a full semester course solely to PhD scholars and each of the students was very unique in many manners. Hence, the expectations and requirements from a course on Research Methodology differed greatly. The fact that many of them were working teachers, many had traditional views about the role of textbook, exams, self study, homework, teachers, and students in a course. I decided to work with the objective of developing their critical thinking to empower them with the attitude of skepticism and the ability to ask penetrating and creative questions with respect to the content, inquiry process, rationale, and evidence in their area. I decided to focus the course on some of the following topics while also keeping each student engaged in his/her own research area
Ways of knowing
Philosophy of Knowledge – Ontological, epistemological, methodological, and axiological questions.
Epistemology of Engineering and Design
Theory vs Empiria,
Disciplines and Interdisciplinarity.
Dimensions of Engineering – basic science, social and business activity, design, practical realisation
Mathematical, scientific, and design thinking for engineering research
Models of Engineering Scholarship –
Curiosity, Questioning, and Problem Formulation,
Conceptualisation, Similarity, Causality, Correlation,
Categorisation, Concept Mapping, Knowledge Structuring
Dilemmas and Ethics wrt engineering research.
Taxonomy of Engineering and Design Research –
Informative (Exploratory/ Descriptive/Explanatory, Basic /Applied),
Analytic-deductive/Quantitative/ Qualitative/Mixed Methods,
Normative (Design and Development),
Performance Metrics. Finding Patterns and Structures,
Reasoning – Induction, Deduction, Abduction, Analogical
Research Design – Fixed vs Flexible.
Ill structured Problem Solving and Decision making
Data Collection –
Techniques and Data Sources,
Data Analysis – quantitative vs qualitative,
Use of IT in data collection and analysis.
Research Thinking – Critical, Systems, Inventive, and Reflective.
Hypothesis formation, Theory building.
Research Validation, Peer Review, Evaluation
Report writing. Dissemination
In the beginning itself, it was clear to me that I will not be able to address all these topics in a single semester. Being a strong supporter of Agile methods in software engineering, I also deliberately avoided preparing a comprehensive and rigid lecture plan as I wanted to explore and evolve the content progressively. Fortunately, my interdisciplinary research on software development education had exposed me to the research methodology literature in education, cognitive psychology, social science, information systems, and also software engineering. In addition to my own research experience, my interest and readings in philosophy of knowledge as well as epistemology also helped me. The above listed topics do not indicate the sequence taken up for discussions in the class or the assignments given to students.
Being a strong advocate and developer of pedagogies of engagement, I decided to use the methods of inductive teaching and active learning that aim to induce learning by engaging students in various activities – in and outside the class, individual as well as group. I formed an e-group on Eduposse.com for assignment announcements, submissions, and discussions. I also invited some guest faculty to join our egroup. Fortunately 10 guest faculty (from India and overseas) including some stalwarts like Prof. Rao Vemuri, Prof. Sartaj Sahni, Prof. Vijay Gupta joined the egroup. In the absence of any general guideline, textbooks, or even recommendations about the possible content and mode of delivery for such a PhD level course in engineering faculty, especially for CSE and ECE students, I planned to use the 3 hrs. long weekly lecture classes for holding interactions and doing individual/small group exercises. The objective was to help them inductively identify and personally rediscover some generic and specific principles related to research and research methodology by engaging them to reflect upon the published research and their own prior experiences.
The students were asked to always carry copies of some important papers referred by them to the class. The in-class exercises were designed around extracting specific information and patterns from these papers. In addition to several in-class exercises in each weekly session, following out-of-class exercises and assignments were announced in the first seven sessions of this course:
Pre course Assignment#1: I hope that you have aspirations to and inclination to do good research through your PhD. You need patience, sincerity and put in hard work. Minimum work, you should put in for good PhD: 5000 hrs. Read https://goelsan.wordpress.com/2010/12/14/will-todays-phd-education-in-india-create-inspiring-academic-leaders-of-tomorrow/. Plan how you intend to put in 5000 hrs. for your PhD work in next few years. Send me your plan.
Pre Course Assignment#2: Download and read a good quality PhD thesis in an area related to your proposed work. Inform me about the thesis (full reference) chosen by you.
Pre Course Assignment#3: Introduce yourself. Mention your interests (including non-academic). Discuss your design/research experience (only mention the projects that took a minimum of 4 months).
Exercise#1: Revisit the already discussed insightful quotes on Philosophy, Science, Inquiry, and Research at https://goelsan.wordpress.com/2013/02/02/research-methodology-iii-some-selected-quotes-about-philosophy-science-inquiry-and-research/. Identify those, if any, that have had any significantly transformational impact on you and/or helped you become clearer about research process. What is the transformation? Keep posting your reflections as comments to this post.
Assignment#1: Review of Research Process in Already Surveyed Literature
Revisit the papers (only journal papers preferably with good citations) already studied by you in the course of Literature Review.
(1) Extract and Tabulate the research method (in domain independent language) used by the researchers for all these papers.
(2) Inductively create a ‘unified model of research process’ using these and also the collective observations made in the class.
(3) Correlate the aspects of your model with the chosen quote discussed in the class.
Exercise#2 : In last class, based on your literature review of minimum 3 papers each, we identified and collated many criteria that have been used by various CSE/IT/ECE researchers to justify the value of their idea. One of you may please post the complete list below. All of you may than continue to expand the list throughout the semester, as and when you come across any other criteria that is not there in the list. Identify if there is a pattern in this.
Assignment#2: Researcher’s Criteria for Evaluating Computing, Electronics, or Communication Ideas
Identify and list the criteria that have been used by researchers in your area wrt the 15 good quality papers (reviewed by you in the last semester or choose new papers). Write a report containing the following:
a) List of papers with full citation and citation count (as per Google Scholar).
b) A data summary in the form of matrix to show the mapping of the criteria to the papers.
c) A note (300-500 word) describing the phenomenon of criteria usage wrt your area of research.
Exercise#3: Good PhD is about becoming a world class (or near world class) Expert in an area.
An ‘Expert’ in a field is someone who demonstrates consistently inspiring and excellent performance. An expert generally knows what to do, based upon mature and practical understanding. Performance standards are well beyond those of most practitioners. Extensive experience with large systems, appreciate subtle and indirect design issues and customer concerns, leads well. High productivity. Learns through apprenticeship to masters, advanced coaching, and development of breadth. Years/decades of experience.
A ‘Master’ in a field is someone who demonstrates the capacity for long range strategic thinking and action. Sees historical drifts and shifting clearings. Has developed a distinctive style. Has produced innovations, altered the course of history in the field. Teaches others to be experts and masters. Develops new methods, admired for long. Learning by working with other masters. Creates and leads professional networks.
Exercise: Create a strategy to become an expert in your area including the following
(i) A plan to occasionally interact with at least one ‘Master’ in your area (or related area) in some manner,
(ii) Joining professional societies like IEEE, ACM, AIS etc. Also identify if there is an active Linkedin group or some other e-group where researchers and/or professionals in your area exchange ideas and share insights. Join such communities. Inform your action as the response to this post.
(iii) Developing the breadth. Post your responses as comments below this post.
Assignment#3: Research Oriented Observations and Measurements in your research area
Identify and list the variables/parameters that have been observed/ measured by the researchers in your area wrt the 15 good quality papers (reviewed by you in the last semester or choose new papers). Write a report containing the following:
a) List of papers with full citation and citation count (as per Google Scholar).
b) A data summary in the form of matrix to show the mapping of the variable/parameters to the papers and the nature of observation/measurement –
(i) computer simulated/ hardware set up in lab/real life and
(ii) Passive observation/collection over space or time/,active probing
c) A note (300-500 word) describing the phenomenon of observation/ measurement in your area of research.
Assignment# 4: Revisit T1: Question 1
IEEE’s tag line is – Advancing Technology for Humanity. a. What is the significance of this tag line wrt CSE/ECE research? b. What are main contemporary challenges for humanity? c. What research trends in your chosen area are trying to meet these challenges? d. How do you intend to contribute to advance these trends through your PhD research?
Assignment# 5: Revisit T1: Question2
Wrt your chosen area of research, write an essay explicating the interrelationship among research process, researcher’s chosen criteria to justify their ideas, and their chosen parameters/variables for observation and measurement (data collection). Your arguments must be supported by the evidence drawn from your literature survey. Give references for cited evidence.
Exercise#4: List the top rated journals and conferences in your chosen areas.
Exercise#5: List some Turing awardees or fellows of IEEE, ACM, AIS, AAAS, NAE, INAE, INSA, etc., whose work in your chosen area has been well recognized as outstanding and exemplary. Also post a short note about the contribution by any 2 of them.
Exercise#6: Identify three examples where some researcher(s) in your area has/have challenged the assumptions of earlier work.
Exercise#7: Identify the phenomenon/parameters in your broader area that are difficult/unethical to observe/measure. Identify what do researchers in your area do to observe/measure such phenomenon/parameters.
Exercise#8: Wrt your chosen research area, list some standardized data sets for researchers that are available in public domain. Write a short note about the evolution and utility of each of these data sets.
Exercise#9: Find out the declared themes of international year by UN in the last 5 years. For example, 2013 has been declared by UN as the year of Water and Governments world-over are encouraging technology projects around water related issues. Identify at least one example of the R&D work in your broad area that is in alignment with these UN theme(s) in the last 5 years.
Assignment# 6: Evolution of Mathematical modeling wrt your research area
Write an essay describing the evolution of mathematical modeling wrt your area. Also highlight the factors that have contributed to such evolution. Cite your references.
Exercise# 10: In your research area, cite examples of ‘fail-safe design’ as a strategy to handle potential failures.
Exercise# 11: In your research area, cite examples of ‘over-design’ as a strategy to handle potential failures.
Exercise# 12: Identify the documented potential risks that influence engineering decisions in your broad area. What do engineers do to manage these risks? Critique five most important papers studied by you in your research area wrt these potential risks.
Exercise# 13: Create an Ontology of knowledge in your focused research area. Based on your literature survey, keep updating your response post every week.
Exercise# 14: Based on your literature survey, write about the relationship between the evolution of designer’s understanding of design constraints and evolution of design of some artifact/system in your area.
Exercise# 15: Transform the five most important papers, so far studied by you in your area, into Concept Maps. Make it a habit of creating concept maps and also updating your ontology after studying every important paper wrt your research.
Exercise# 16: Study the code of ethics of ACM, IEEE, and also ACM-IEEE-CS code of ethics for Software Engineers.
Exercise#17: Prepare your action plan for deliberate practice in this semester in order to enhance few of the following subset of research skills. How will you monitor your progress?
A1. Demonstrate the ability to recognize and validate problems
A1.1 Able to define research problems from a coherent analysis of gaps in existing knowledge base.
A1.2 Able to identify areas where investigation might produce new knowledge.
A1.3 Able to write a research proposal, describing research questions, context, sources and methodology to the level required of applications for postdoctoral work.
A2. Demonstrate original, independent and critical thinking, and the ability to develop theoretical concepts
A2.1 Able to formulate hypotheses and/or research questions for the purposes of designing a personal research project.
A2.2 Able to provide new and innovative research ideas.
A2.3 Able to objectively and knowledgeably criticize published research.
A3. Demonstrate a knowledge of recent advances within one’s field and in related areas
A3.1 Can communicate knowledgeably about their research topic with supervisor and peers, debating concepts.
A3.2 Familiar with recent relevant literature.
A3.3 Can write a literature review of publication standard on the topic.
A4. Demonstrate an understanding of relevant research methodologies and techniques and their appropriate application within one’s research field.
A4.1 Able to discuss and prioritise a range of methodologies to address a research question.
A4.2 Has in depth knowledge and understanding of appropriate techniques and their application.
A5. Demonstrate the ability to critically analyse and evaluate one’s findings and those of others
A5.1 Ability to objectively acknowledge weaknesses and assumptions in one’s findings.
A5.2 Ability to apply the same objectivity to the work of others. Good understanding of appropriate methods for testing conjectures or tentative conclusions.
A5.3 Excellent IT ability in data collection analysis and presentation in an appropriate graphical form.
A6. Demonstrate an ability to summarise, document, report and reflect on progress
A6.1 Able to verbally summarise a research problem succinctly to different audiences.
A6.2 Able to objectively criticise own research and define future work.
A6.3 Able to produce written summaries of a variety of lengths to suite the purpose.
A6.4 Able to write progress reports on research of an appropriate professional standard.
(Ref: Research Councils, Joint Statement of the Research Councils’/Ahrb’s Skills Training Requirements for Research Students. 2001, Research Councils. Cited by A. P. Bromley*, J. R. Boran and W. A. Myddelton 92005), Self Assessment of Skills by Research Postgraduates – An Analysis of Initial Results, University of Manchester, pp 17-19, UK)
Assignment#7: Epistemological and Axiological Perspective for Literature Review
Re-review any 5 high quality papers in your area already studied by you with epistemological and axiological questions. Reflecting upon these reviews, write a short essay (upto 400-500 word) about the value of epistemological and anxiological inquiry in research. Use the systematic approach for reflection (What? So what?,Now what?) already discussed in the class.
Assignment#8: Video lecture to introduce your research area
Prepare a 3 minute video lecture giving a contemporary introduction to your area of research in scholarly acceptable and interesting ways. Do not prepare a Powerpoint presentation. You can edit the recorded video to integrate graphics and animation. You can upload your videos on web and post the links here.
Exercise#18: Prepare video recordings of 5-25 minute each to demonstrate the problem solving process for any two CS/ECE problems wrt any UG/PG course closely related to your research area. Choose interesting problems and demonstrate interesting ways of solving these problems.
Regularly show me your plan and/or videos for the feedback. You can upload your videos on web.
Assignment# 9: Experiments in your research area
Summarize 5 experiments (not simulations) performed recently by some researchers in your area. Reflect (Use What?, So what?, Now what? model for reflection) on these summaries to write a short essay.
The course was planned as a full semester 3 credit course. This would have taken around 14-15 weekly classroom sessions of 3 hrs each. However, for some administrative reasons, the course was stopped midway after seven sessions. Hence, a large number of initially planned topics listed above could not be taken up for discussion. Hopefully few research scholars enjoyed this incomplete course and developed a slightly clearer perspective and understanding about research, their problem, and methodology. It surely gave me an opportunity to creatively reflect upon my own experiences and readings as well as make my very humble contributions to help increase the quality of doctoral education in engineering disciplines in India. Perhaps there will be more occasions and forums to further develop, refine, and consolidate this model.
I also hope that some other PhD scholars will also find these exercises and assignments useful for their own research work. I shall be very enthusiastically looking forward to the feedback and suggestions for improving above listed exercises and assignments as well as for designing many more wrt the remaining topics.
You may also like to see: