1. No category

COURSEMANUAL
EngagedandResponsibleScholarship
May19,2014
RenéBekkers
FreekColombijn
PeterGroenewegen
IvarVermeulen
IngridWakkee
PhDprogramme2013‐2014
GraduateSchoolofSocialSciences
VUUniversityAmsterdam
EngagedandResponsibleScholarship
PhDprogrammeGraduateSchoolofSocialSciences,2013‐2014
VUUniversityAmsterdam
Period:
PartA(period6,June2014),partB(period1,Sept.2014)
CourseCredits:
3/6ECTS(3foreachpart)
Lecturers:
Prof. René Bekkers, Dr. Ivar Vermeulen, Dr. Freek Colombijn,
Prof.PeterGroenewegen,Dr.IngridWakkeeandothers
Fee(externalparticipants):€300,‐/600,‐
Schedule:
Sessions Date
Time
Location
PartA
1
Tuesday,June10
13.00‐15.00
HG‐5A10
2
Monday,June16
13.00‐17.00
HG‐5A11
3
Tuesday,June17
13.00‐17.00
HG‐5A11
4
Thursday,June26
09.00‐15.00
HG‐6A13
PartB 5
Friday,Sept.5
13.00‐17.00
TBA
6
Friday,Sept.12
13.00‐17.00
TBA
7
Friday,Sept.19
13.00‐17.00
TBA
8
Friday,October3
13.00‐17.00
TBA
Courseobjectives
Thiscourseaimstoequipparticipantswithaheightenedsenseofawarenessofwhatittakestobe
andbecomeasuccessfulandresponsiblesocialscientificresearcher.
Thefirstpart(partA)ofthecoursefollowsfromeffortsoftheFacultyofSocialSciencestoprevent
violations of research integrity. It seeks to contribute to a reflection and discussion on the
normative consequences of the abstract ideals of science and an awareness of standards of good
conductamongPhDcandidatesinthesocialsciences.
Recentscandalsinthesocialscienceshaveputintegrityinresearchattheheartofmuchdebatein
thepressandintheacademiccommunity.Thehighlyvisiblecasesinsocialpsychology,economics
andanthropologymayonlybethetipoftheiceberg.Violationsofresearchintegrityincludeawide
rangeofbehaviors,fromseeminglyminorproblemsresultingfrom‘sloppyscience’,toquestionable
research practices (QRP), research misconduct (RM), abuse of power by dissertation supervisors,
andoutrightfabrication,fraud,andplagiarism(FFP).Howcansuchviolationsofresearchintegrity
be prevented? What should researchers do when they notice violations of research integrity?
Formulated in a more positive way: how can researchers contribute to responsible social science
research?
VUGraduateSchoolofSocialSciences
1
Coordinated efforts to promote responsibility originate from self‐regulation by the academic
communitythroughcodesofconductformulatedbyNationalAcademiesofSciencesuchasNWO,
academic journals, professional associations, universities, and faculties. Recently, funders such as
theEuropeanCommissionhaveincreasedeffortstopromote responsibilityinresearchbyadding
fundingallocationcriteria.Atthesametime,theincentivesforindividualresearchersembeddedin
tenuretracks,academiccareers,publicationcultures,aretemptingresearchersinthedirectionof
violationsofintegrity.ThiscoursehelpsPhDcandidatesinthesocialsciencestodevelopaposition
vis‐à‐vis the conflicting demands regarding research integrity. How should researchers deal with
them?Thesecondpartofthecourse(partB)stimulatesparticipantstoreflectonthefactorsand
criteriathatmaydetermineone’spathandsuccessindevelopingacareerasascientificresearcher,
eitherinacademiaorappliedindustries.
Because of the importance of responsibility and integrity in social science research, and of
individual responsibility in determining one’s career path, this course is an essential part of the
educationforallSocialSciencePhDcandidates.
Bytheendofthiscourse,participants
1. are aware of the values embodied in science, according to the association of universities
(VSNU);
2. are aware of the behavioral consequences of these values in daily practice, and have
acquiredtheknowledgeforproperscientificbehavior;
3. are aware of the internal review board procedures and quality control procedures within
theFacultyofSocialSciencesatVUUniversity;
4. are aware of the codes of conduct developed to prevent integrity violations by research
funders,professionalassociations,leadingscholars,andacademicjournals;
5. havedevelopedacriticalpositionontheirownresponsibilityinacademicresearch;
6. have developed an awareness of the factors that may influence their future career as a
socialscientist(inacademiaorappliedindustry);
7. havedevelopedanawarenessofthecareerpaththeyaspire.
Coursecontent
Thecourseconsistsoftwoparts,whichcanbefollowedasindependentmodules(3ECeach).
PartAdealswithintegrityinresearchconductandhandlingofdataanditsrequirementsaccording
tofaculty,university,nationalanddisciplinaryagreements.Youwillexploretheterrainofscientific
integrity and research quality, discuss violations of integrity, sloppy science, and questionable
researchpractices.Youwilllearnaboutproperdatacollectionandstorage;reliableandverifiable
publicationpractices,impartiality,independenceandnormsonco‐authorship.
VUGraduateSchoolofSocialSciences
2
PartBfocusesonyourroleandbehaviorasasocialscientist,bothinacademiaandin(collaboration
with)appliedindustries.Youwilldiscusscriteriaforsuccessfulcareerswithindifferentdisciplines,
andthefactorsthatshapereputationsandpositionsthatyoumayachieveasascientificresearcher.
Thiswillmakeyoumoreaware ofthefactorsthatmayinfluencethedevelopment of your(non‐)
academiccareer.
Methods
Thisisanintensivecourse.InPartA,participants(1)readpapersfromtherecentandongoing
debatesonresearchintegrityinthesocialsciences;(2)writeapaperreflectingonanissueoftheir
choiceregardingintegrityandqualityofresearch;(3)presenttheirpaper;(4)commentontwo
papersbyotherparticipants;(5)participateinplenarydiscussionsduringtheworkshopmeetings.
InPartB,participants(1)writetwobriefessays(inblogform)reflectingonissuesthatare
importantwhendevelopingascientificcareer,(2)fulfilltwoassignmentsreflectingonissueswhen
developinganappliedcareerpath,(3)participateinplenarydiscussionsduringthemeetings.
Formsoftuition
Coursemeetingsareamixbetweenlecturesandworkshops.
Courseaudience
Thiscourseisopentoboth‘regular’andexternalPh.D.candidates(‘buitenpromovendi’)whoare
linkedtothevariousdepartmentsoftheVUFacultyofSocialSciences.Thecourseisstrongly
advisedforcurrentPhDcandidates,andwillbemandatoryforPhDcandidatescommencingin
September2014.PhDcandidatesfromrelevantotherVUfaculties(i.e.,psychology,economicsand
managementscience)andotheruniversitiesarewelcomeifspaceallows.Theseexternal
participantsarechargedafee(seeabove).
Registration
PleaseregisterforthiscoursebyemailtoSaskiaJans([email protected])oneweekpriortothefirst
meetingofthecourse.ShewilladdyournametotheBlackboardpageofthiscoursesoyouget
accesstoallnecessarydocumentsandupdates.Alsoultimatelyoneweekbeforethestartofthe
courseparticipantsarerequestedtosubmitanabstractoftheirPhDproject(onepagemax)to
SaskiaJans.
Assessment
ToobtainECTScreditsforthecourseparticipantsarerequiredto(1)bepresentandactively
participateinallsessions;(2)sufficientlyfulfillallassignmentsrequiredintherelevantmodule
(partA,B;seebelow).
VUGraduateSchoolofSocialSciences
3
DetailedSchedule
Foreachsessionthereadingsandassignmentsareindicated.Inpreparationofthecourseplease
readthearticlesasindicatedbelow.
PARTA(June2014)
SessionA1:Anexplorationofintegrity
andresearchquality(RenéBekkers)
Usingadilemmagame,weexplorethe
terrainofintegrityandresearchquality.
 Plenarydiscussionofdilemmas:
violationsofintegrityandbadscience
 Individualsuggestionsforresearch
questionsforapaperreflectingona
dilemmaincurrentresearchpractice
AssignmentA1:FormulatePreliminary
ResearchQuestion
Date
Tuesday,
June10,
13.00–
15.00,
HG‐
5A10
Tuesday,
June10,
13.00
Monday,
June16,
13.00–
17.00,
SessionA2:Integrityandqualitynormsin
socialscienceresearch(RenéBekkers)
 Plenarylecture:
o Values:scrupulousness,reliability,
HG‐
verifiability,impartiality,and
5A11
independence
o Codesofconduct:International,
European,disciplinary,nationaland
local
 Plenarydiscussion:
o Qualityofworkandyourjudges
o Publicationpracticesandnormsonco‐
authorship
o ThePhDcandidate–supervisor
relationandotherinteractionswith
seniors
o Reviewandjudgmentofworkofothers
AssignmentA2:RefineResearchQuestion. Monday,
June16,
13.00
VUGraduateSchoolofSocialSciences
Readings
Alper(2008)
Bekkers(2012,2013)
Hartmann(2014)
Huberts(2014)
KNAW(2013a)
KNAW(2013b)
SCA(2013)
VSNU(2012)
4
SessionA3:Integrityandqualityofdata
management,analysis,andreporting(Ivar
VermeulenandFreekColombijn)
 Plenarylecture:
o Researchethics
o Relationwithresearchparticipants
o Datacollectionandstorage
o Describingandusingdata
o Questionableresearchpractices
 Plenarydiscussion:
o Questionableresearchpractices
Tuesday,
June17,
13.00–
17.00,
HG‐
5A11
VandePort(2011)
Clifford(1986)
Rosaldo(1986)
Tomlinson(2011)
Matelski(inpress)
John,Loewenstein&Prelec(2012)
Klein,R.etal.(inpress)
Masicampo&Lalande(2012)
Nosek&Bar‐Anan(2012)
Nosek,Spies&Motyl(2012)
Simmons(2012)
Simmons,Nelson&Simonsohn
(2011)
Wagenmakers,Wetzels,Borsboom,
VanderMaas&Kievit(2012)
Wicherts,J.M.,Bakker,M.,&
Molenaar,D.(2011)
AssignmentA3:PeerReview
SessionA4:Conference
 Plenarypresentations:
o Papersummary
o Peerrefereereports
o Commentsandquestionsfromthe
audience
AssignmentA4:Revisepaper
Thursday
June26,
09.00‐
15.00,
HG‐
6A13
PARTB(September‐October2014)
SessionB1:Thecharacterofsocialscience
asasocialactivity(PeterGroenewegen,
PetervandenBesselaar)
 Thesocialdynamicsofacademiawithin
scientificfields
 Scientificcommunitiesand
communication
 Elementsofcommunity(peerreviewfor
proposals,journalsandbooksetc)in
socialscience
VUGraduateSchoolofSocialSciences
Friday,
Sept.5,
tba
13.00–
17.00
5


QualityevaluationinDutchsocialscience
Factorsthatdeterminesuccessand
reputationinacademia
AssignmentB1:CareersinacademiaI
SessionB2:Careersinsocialscience(Peter Friday,
Sept.12
Groenewegen,MarjoleinBroesevan
Groenou)
 Socialstructureofscience:statuspro’s
andbenefits
 Careerstructureconcretestepsand
advice
 Normsofyourowndiscipline(sociology,
socialpsychology,communication
scienceanthropologyetc.)
AssignmentB2:CareersinacademiaII
SessionB3:CareersbeyondAcademia
(IngridWakkee)
 Nonuniversitycareers
 Applyingforthirdpartygrants,EU
fundingetc.
 Nonresearchcareers
 (Academic)skillsandhowtopresent
theminyourcv.
AssignmentB3:Writingyourresume
Friday,
Sept.19
Friday,
SessionB4:Workinginapplied
October
collaborativeprojects(IngridWakkee)
3
 Workinginappliedprojects
o Goalsettinganddivisionofworkload
o Responsibilityandaccountability
 Dealingwithexternalstakeholders
o Organizationalissuesanddealing
withtargetsanddeadlines
o Packaging’andCommunicationof
researchresultsintopracticaladvise
 Consultancyandcontractresearch
o Sidelinesorpartofthejob?Single
trajectoryordevelopmentof
methodologies
VUGraduateSchoolofSocialSciences
tba
Lam(2007)
Lam(2010)
Benneworth&Jongbloed(2010)
Boardman&Ponomariov(2009)
Olmos‐Peñuela,Castro‐Martínez,&
D’Este(2014).
6
AssignmentB4:Proposalforapplied
collaborativeresearchproject
Assignments
PARTA
o AssignmentA1:FormulatePreliminaryResearchQuestion
Inthisfirstassignmentyoupresentapreliminaryformulationoftheresearchquestionthatyouwill
answerinyourconferencepaper.
o AssignmentA2:RefineResearchQuestion
Refinetheresearchquestionofyourconferencepaperbasedonthesuggestionsofthecourse
coordinators.Selecttwoothercourseparticipantswhowillactaspeerreviewersofyourpaperand
agreeuponadeadlineforsubmissionofyourpaper.
o AssignmentA3:Peerreview
Ataconvenienttimeyoudiscussthefirstdraftsoftwoconferencepaperswithotherparticipants.
o AssignmentA4:Revisepaper
Buildingonthecommentsfromtheotherparticipantswhopeerreviewedyourpaperand
suggestionsyoureceivedattheconferencepresentationyoureviseyourpaperandsubmitittothe
coursecoordinators.
PARTB
o AssignmentB1:CareersinacademiaI
Interviewoneoftheseniorresearchersinyourdepartmentonthefactors,conditionsand
motivationalrequirementsthatdeterminesuccessindevelopingascientificcareer.
WriteablogpostontheseissuesfortheSocializingSciencewebsite.
o AssignmentB2:CareersinacademiaII
Identify relevant criteria for the evaluation of publication quality and criteria for hiring someone
whojustfinishedhis/herdissertationinyoursubdiscipline;thinkaboutyourownworkandplans,
anddiscussthesewithsomeonewhodefendedhisorherdissertationinthelast3years.Writea
blogpostontheseissuesfortheSocializingSciencewebsite.
o AssignmentB3:Writingyourresume
Reorganizeyourcurrentresumeandwritealetterofmotivationinordertoimpressahypothetical
non‐academicemployer.Yourletterandresumewillbeevaluatedbyanexternalreviewerwith
experienceinhiringscientistforappliedresearchprojects.
o AssignmentB4:Proposalforanappliedcollaborativeresearchproject
Writearesearchproposalforapotentialexternalstakeholderinwhichyoumotivatewhatyoucan
doforhim/her.Ifyouhavealreadybuiltsomeexperienceinworkingintheseareaspleasecritically
VUGraduateSchoolofSocialSciences
7
evaluatetheseexperiencesbasedonsuggestionsfromthecourse.Ifyouareyettoengageinyour
firstcollaborativeprojectformulateashortprojectproposaltosuchanendinwhichyouaddress
theissuesdiscussioninthesession.
Readings
PartA
Meeting2
o Alper,H.(2008).Howcanresearchintegritybebestachieved?Materialstoday,11(4):60.
o Bekkers,R.(2012).RiskFactorsforFraudandAcademicMisconductintheSocialSciences.
http://renebekkers.files.wordpress.com/2012/11/risk‐factors.pdf
o Bekkers,R.(2013).HowIncentivesLeadusAstrayinAcademia.
http://renebekkers.files.wordpress.com/2013/02/how‐incentives‐lead‐us‐astray‐in‐
academia2.pdf
o Hartmann,T.(2014,24February).ResearchIntegrity:Good‐PracticeChecklist.VUDepartment
ofCommunicationScience,Amsterdam.
o Huberts,L.(2014).WetenschappelijkeintegriteitenFSW.MemofortheAHO,9May2014.
o ErasmusUniversityRotterdam(2013).DilemmaGame:ProfessionalismandIntegrityin
Research.Rotterdam:EUR.
o KNAW(2013a).ResponsibleResearchDataManagementandthePreventionofScientific
Misconduct.Amsterdam:RoyalNetherlandsAcademyofArtsandSciences.
o KNAW(2013b).TrustinScience,SummaryofKNAWreport‘Vertrouwenindewetenschap’
(https://www.knaw.nl/en/news/publications/vertrouwen‐in‐
wetenschap/@@download/pdf_eng_file/Summary‐advies‐vetrouweninwetenschap_web.pdf)
o SocialandCulturalAnthropologyVUUniversityAmsterdam(2013).Protocolvoorinteger
onderzoek/ProtocolfortheIntegrityofResearch
http://www.fsw.vu.nl/nl/Images/Protocol%20voor%20het%20voorkomen%20van%20fraud
e%20bij%20SCA_tcm30‐365845.pdf
o VSNU(2012).DeNederlandsegedragscodewetenschapsbeoefening:principesvangoed
wetenschappelijkonderwijsenonderzoek./TheNetherlandsCodeofConductforScientific
Practice:PrinciplesofGoodScientificTeachingandResearch.Amsterdam:VSNU.
Meeting3:FreekColombijn
o Clifford,J.(1986).Introduction;partialtruths.Pp.1‐26in:JamesCliffordandGeorgeE.Marcus,
Writingculture:Thepoeticsandpoliticsofethnography.Berkeleyetc;UniversityofCalifornia
Press.
o Matelski,M.,(inpress).Onsensitivityandsecrecy:Howforeignresearchersandtheirlocal
contactsinMyanmardealwithriskunderauthoritarianrule,JournalofBurmaStudiesvol.18(1)
o Rosaldo,R.(1986).Fromthedoorofhistent:Thefieldworkerandtheinquisitor.Pp.77‐97in:
JamesCliffordandGeorgeE.Marcus,Writingculture:Thepoeticsandpoliticsofethnography.
Berkeleyetc;UniversityofCaliforniaPress.
o Tomlinson,K.(2011).Theanxietiesofengaginginmulti‐sitedPhDresearch:Reflectionson
researchindigenousrightsprocessesinVenezuela.In:SimonColeman&PaulinevonHellerman
VUGraduateSchoolofSocialSciences
8
o
(eds),Multi‐sitedethnography:Problemsandpossibilitiesinthetranslocationofresearch
methods,pp.161‐173.NewYorkandAbingdon:Routledge,2011.
VandePort,M.(2011).Ecstaticencounters:BahianCcandombléandthequestforthereallyreal,
Amsterdam:AmsterdamUniversityPress.IntroductionandConclusion(pp.11‐44and249‐
261).
Meeting3:IvarVermeulen
o John,L.K.,Loewenstein,G.,&Prelec,D.(2012).Measuringtheprevalenceofquestionable
researchpracticeswithincentivesfortruth‐telling.PsychologicalScience.
o Klein,R.etal.(inpress)InvestigatingVariationinReplicability:A“ManyLabs”Replication
Project.SocialPsychology.
o Masicampo,E.,&Lalande,D.R.(2012).APeculiarPrevalenceofPValuesJustBelow.05.The
QuarterlyJournalofExperimentalPsychology,65(11),2271‐2279.
o Nosek,B.A.,Spies,J.R.,&Motyl,M.(2012).Scientificutopia:II.Restructuringincentivesand
practicestopromotetruthoverpublishability.PerspectivesonPsychologicalScience,7,615–
631.
o Nosek,B.A.&Bar‐Anan,Y.(2012).ScientificUtopia:I.Openingscientificcommunication.
PsychologicalInquiry.
o Simmons,J.P.,Nelson,L.D.,&Simonsohn,U.(2011).False‐positivepsychology:Undisclosed
flexibilityindatacollectionandanalysisallowspresentinganythingassignificant.Psychological
Science,22,1359‐1366.
o Simmons,J.P.,Nelson,L.D.,&Simonsohn,U.(2012).A21WordSolution.Dialogue,TheOfficial
NewsletteroftheSocietyforPersonalityandSocialPsychology,26(2),4‐7.
o Wagenmakers,E.–J.,Wetzels,R.,Borsboom,D.,&vanderMaas,H.L.J.(2011).Why
psychologistsmustchangethewaytheyanalyzetheirdata:Thecaseofpsi:CommentonBem
(2011).JournalofPersonalityandSocialPsychology,100,426‐432.
o Wicherts,J.M.,Bakker,M.,&Molenaar,D.(2011).Willingnesstoshareresearchdataisrelated
tothestrengthoftheevidenceandthequalityofreportingofstatisticalresults.PLoSOne,6,
e26828.doi:10.1371/journal.pone.0026828
Recommended
o Abma,R.(2013).DePublicatiefabriek.OverdebetekenisvandeaffaireStapel.Nijmegen:
Vantilt.
o AMC/VUmcEditorialTeam(2013).Independenceinscientificresearch:AMC/VUmcResearch
Code.Amsterdam:AMC/VUmc.
o Bouter,L.(2014).Perverseprikkelsofrotteappels?/Perverseincentivesorrottenapples?
Inaugurallecture,VUUniversityAmsterdam.
o DellaPorta,D.&Keating,M.(2008).Howmanyapproachesinthesocialsciences?An
epistemologicalintroduction.In:M.KeatingandD.DellaPorta(Eds),Approachesand
methodologiesinthesocialsciences:Apluralistperspective,pp.19‐39.CambridgeandNewYork:
CambridgeUniversityPress.
o ESF(2010).FosteringResearchIntegrityinEurope.Strasbourg:ESF.
VUGraduateSchoolofSocialSciences
9
o
o
o
o
o
ESF/ALLEA(2011).TheEuropeanCodeofConductforResearchIntegrity.
Strasbourg/Amsterdam:ESF/ALLEA.
Martin,B.R.(2013).Whitherresearchintegrity?Plagiarism,self‐plagiarismandcoercive
citationinanageofresearchassessment.ResearchPolicy,42:1005‐1014.
NedLebow,R.(2007).Socialscienceasanethicalpractice.JournalofInternationalRelations
andDevelopment10:16‐24.
Pittaway,E.,Bartolomei,L.,&Hugman,R.,2010,‘Stopstealingourstories’:Theethicsof
researchwithvulnerablegroups’,JournalofHumanRightsPractice,2(2):229‐251.
Respectproject(2004).RespectCodeofPracticeforSocio‐EconomicResearch.Brighton:
Respectproject.
o
Simonsohn,U.,Nelson,L.D.&Simmons,J.P.(2013)P‐curve:Akeytothefiledrawer.
JournalofExperimentalPsychologyGeneral,inpress.
o
Tyler,S.A.(1986),‘Post‐modernethnography:Fromdocumentoftheocculttooccult
document’,in:J.CliffordandG.E.Marcus(Eds),Writingculture:Thepoeticsandpoliticsof
ethnography,pp.122‐140.Berkeleyetc;UniversityofCaliforniaPress.
VanderHeijden,A.J.,Groenen,P.J.F.,Zeelenberg,R.&TeLindert,R.(2014).Reportofthe
SmeestersFollow‐UpInvestigationCommittee.Rotterdam:EUR.
Wager,L.&Kleinert,S.(2011).Responsibleresearchpublication:internationalstandardfor
authors.Apositionstatementdevelopedatthe2ndWorldConferenceonResearchIntegrity,
Singapore,July22‐24,2010.Pp.309‐316in:Mayer,T.&Steneck,N.(Eds).PromotingResearch
IntegrityinaGlobalEnvironment.ImperialCollegePress/WorldScientificPublishing,
Singapore.
o
o
PartB(completelisttobeannounced)
o W.H.Starbuck(2006),TheproductionofKnowledge,OxfordUniversityPress.
o Bakanic,V.,McPhail,C.,&Simon,R.J.(1987).TheManuscriptReviewandDecision‐Making
Process.AmericanSociologicalReview,52(5):631‐642.
o Benneworth,P.&Jongbloed,B.(2010)Who matters to universities? A stakeholder perspective
on humanities, arts and social sciences valorisation,HigerEducation,59(5):567‐588
o Boardman,P.C.&Ponomariov,B.(2009).UniversityResearchersworkingwithprivate
companies.Technovation,29(20:142‐153
o Hargens,L.(1988).Scholarlyconsensusandjournalrejectionrates.AmericanSociological
Review,53(1),139‐151.
o Jacobsen,C.B.,&Andersen,L.B.(2014).PerformanceManagementforAcademicResearchers:
HowPublicationCommandSystemsAffectIndividualBehavior.ReviewofPublicPersonnel
Administration,34(2):84‐107.
o Lam,A.(2007).Knowledgenetworksandcareers:Academicscientistsinindustry–university
links.JournalofManagementStudies,44(6):993‐1016.
o Lam,A.(2010).From‘ivorytowertraditionalists’to‘entrepreneurialscientists’?Academic
scientistsinfuzzyuniversity—industryboundaries.SocialStudiesofScience,40(2):307‐340.
o Olmos‐Peñuela,J.,Castro‐Martínez,E.,&D’Este,P.(2014).Knowledgetransferactivitiesin
socialsciencesandhumanities:Explainingtheinteractionsofresearchgroupswithnon‐
academicagents.ResearchPolicy,43(4):696‐706.
VUGraduateSchoolofSocialSciences
10