1. No category

La valutazione dei corsi di dottorato, la VQR
2011-2014 e introduzione a ORCID
Sergio Benedetto
Consiglio Direttivo ANVUR
[email protected]
CRUI, 12 gennaio 2015
Summary
• The evaluation of PhD programmes
• Statistics on Italian PhD programmes
• VQR 2011-2014: The ANVUR perspective
• Introduction to ORCID
2
Summary
• The evaluation of PhD programmes
• Statistics on Italian PhD programmes
• VQR 2011-2014: The ANVUR perspective
• Introduction to ORCID
3
The evaluation of PhD programmes
The criteria for the evaluation of PhD programmes are listed in
article 13 of DM 45:
1. … The MIUR contributes annually to the funding of PhD
programmes.
2. The MIUR funds are assigned annually by a Ministry Decree, after
listening to ANVUR opinion, based on the following criteria:
V1) Quality of research of members of the Doctoral Body;
V2) Degree of internalisation of the PhD programme;
V3) Degree of collaboration with the socio-economical system;
V4) Attractiveness of the PhD programme;
V5) Availability of services, infrastractures, financial resources for the
PhD students;
V6) Career opportunities of PhD students after the degree
4
The evaluation of PhD programmes
• In additon to previous criteria, ANVUR will also consider the following
criterion:
V7) The scientific activity of PhD students during and after the PhD
programme as documented by scientific publications of PhD students
and post PhDs during the PhD programme and in the three following
years
5
The evaluation of PhD programmes
• Since criteria V1-V7 depend on the scientific area of reference of PhD
programmes, ANVUR will form separate rankings for each of the 16
scientific areas defined in VQR 2004-2010, i.e., the 14 areas CUN of which
two (area 8 and area 11) subdivided into two subareas 8a, 8b, 11a and 11b
• To the criteria V1-V7 a suitable indicator IV1-IV7 will be associated, which
depends upon the area, the institution, and the PhD programme.
6
The evaluation of PhD programmes
• Since criteria V1-V7 depend on the scientific area of reference of PhD
programmes, ANVUR will form separate rankings for each of the 16
scientific areas defined in VQR 2004-2010, i.e., the 14 areas CUN of which
two (area 8 and area 11) subdivied into two subareas 8a, 8b, 11a and 11b
• To the criteria V1-V7 a suitable indicator IV1-IV7 will be associated, which
depends upon the area, the institution, and the PhD programme.
7
The evaluation of PhD programmes
Indicator IV1: Quality of research of members of the Doctoral Body
Three sub-indicators will be used to construct IV1:
1. The indicator RVQR of VQR 2011-2014 of the Doctoral Body
2. The indicator XVQR of VQR 2011-2014 of the Doctoral Body
3. The indicator of scientific activity IAS in the last five years, different for
the bibliometric areas (1-7, 8a, 9, 11b) and non-bibliometric areas (8b,
10, 11a, 12, 13, 14). For each member of the Doctoral Body five
research outcomes will be considered, published in the five years
preceding the evaluation
8
The evaluation of PhD programmes
• For the biblio areas the indicator IAS measures the impact of the journals
where the articles had been published. It is computed as the sum of the
ratios between the value of the impact indicator of each publication (IF,
Article influence, Eigenfactor, SJR, SNIP, …) and its value averaged within
the Science Category where the journal belongs, divided by five times the
number of members of the Doctoral Body
• For the non biblio areas the indicator IAS consists in the evaluation of
publications of each member of the Doctoral Body, giving preference in
the order to books reviewed in journal belonging to the class A of ASN; to
book chapters reviewed in journal belonging to the class A of ASN, to
articles published in journals belonging to the class A of ASN
•
The indicator is computed as the ratio between the weighted sum of the
publications of members of the Doctoral Body divided by five times the
number of members, and the weighted sum of the publications of all
members of all Doctoral Bodies of the same area divided by five times their
number
9
The evaluation of PhD programmes
The final indicator IV1 of the k-th PhD programme of Institution j within area i
is computed as:
IV1i,j,k=α1RVQR + α2 XVQR + α3IAS
with the following values of the weights: α1=0.3, α2 =0.3, α3=0.4
10
The evaluation of PhD programmes
Indicator IV2: Degree of internalisation of the PhD programme
Three sub-indicators will be used to construct IV2:
1.
The ratio IME between the percentage of PhD students who have spent a
documented period of at least one month in a foreign university or
laboratory and the percentage of PhD students who have spent a
documented period of at least one month in a foreign university or
laboratory averaged within the PhD programs of the same area
2.
The ratio IST-VP between:
– The sum of the overall number of periods with duration greater than 15 days
spent officially as Visiting Professor/Visiting Scholar or Visiting Scientist in foreign
universities or research centres by members of the Doctoral Body, and the overall
number of periods with duration greater than 15 days spent in the university
hosting the PhD programme (or in ones of the universities of the PhD consortium)
by foreign members of the Doctoral Body in the last 5 years, and
– The number of the same periods averaged within the PhD programmes of the
same area
11
The evaluation of PhD programmes
3. a) The third sub-indicator for the biblio areas is the ratio IST-VQR
between the average number of publications of members of the
Doctoral Body with “foreign” coauthor that have been assigned the
class “excellent” in the VQR 2011-2014, and the same number
averaged within the PhD programmes of the same area
3. b) The third sub-indicator for the non biblio areas is the best
among the two following indicators:
– The ratio IMORE between the average number of books published outside
Italy and reviewed by journals belonging to class A (ASN) by members of
the Doctoral Body and the same number averaged within the PhD
programmes of the same area
– The ratio IAE between the average number of articles listed in the PhD
programme proposal published by members of the Doctoral Body in
journals in Class A (ASN) ISI-Scopus with foreign editors and the same
number averaged within the PhD programmes of the same area
12
The evaluation of PhD programmes
The final indicator IV2 of the k-th PhD programme of Institution j within area i
is computed as
IV2i,j,k=β1 IST-VQR(or β1 IMORE or IAE for non biblio) + β2IME + β3IST-VP
with the following values of the weights: β1=0.3, β2 =0.4, β3=0.3
13
The evaluation of PhD programmes
Indicator IV3: Degree of collaboration with the socio-economical
system
Two sub-indicators will be used to construct IV3:
1. The ratio IBD between the percentage of PhD fellowships funded by
institutions private or public (in this latter case operating on the
whole national territory) and the same number averaged within the
PhD programmes of the same area
2. The ratio IBR of patents having PhD students as co-inventors averaged
wrt the number of PhD students of the PhD programme and the same
number averaged within the PhD programmes of the same area
14
The evaluation of PhD programmes
The final indicator IV3 of the k-th PhD programme of Institution j within area i
is computed as
IV3i,j,k=γ1IBD + γ2IBR
with the following values of the weights: γ1=0.7, γ2 =0.3
15
The evaluation of PhD programmes
Indicator IV4: Attractiveness of the PhD programme
Two sub-indicators will be used to construct IV4:
1. The ratio ICE between the percentage of PhD Italian candidates who
got their “Laurea” degree in a different university and the average
percentage within the PhD programmes of the same area
2. The ratio ICS between the percentage of PhD “foreign” candidates
(i.e., who got their “Laurea” degree in a foreign university) and the
average percentage within the PhD programmes of the same area
16
The evaluation of PhD programmes
The final indicator IV4 of the k-th PhD programme of Institution j within area i
is computed as
IV4i,j,k=δ 1ICE + δ 2ICS
with the following values of the weights: δ1=0.7, δ2 =0.3
17
The evaluation of PhD programmes
Indicator IV5: Availability of services, infrastractures, financial
resources for the PhD students
Since criterion V5 refers to context data that have been already
considered duing the accreditation phase, no indicator will be
associated to the criterion
18
The evaluation of PhD programmes
Indicator IV6: Career opportunities of PhD students after the
degree
Two sub-indicators will be used to construct IV6:
1. The percentage of PhD graduates being employed after 2 (ISB2) years
2. The percentage of PhD graduates being employed after 5 (ISB5) years
Both indicators will be obtained from the extensive analysis, involving all
PhD graduates, carried out by ISTAT
19
The evaluation of PhD programmes
The final indicator IV4 of the k-th PhD programme of Institution j within area i
is computed as
IV4i,j,k=η1ICE + η2ICS
with the following values of the weights: η1=0.7, η2 =0.3
20
The evaluation of PhD programmes
Indicator IV7: The scientific activity of PhD students during and
after the PhD programme as documented by scientific
publications of PhD students and post PhDs during the PhD
programme and in the three following years
For both biblio and biblio areas up to three publications per PhD
or post PhD student are considered
The indicator is computed in a different way for biblio and non
biblio areas
21
The evaluation of PhD programmes
• For the biblio areas the indicator measures the impact of the journals
where the articles had been published. It is computed as the sum of the
ratios between the value of the impact indicator of each publication (IF,
Article influence, Eigenfactor, SJR, SNIP, …) and its value averaged within
the Science Category where the journal belongs, divided by three times
the number of PhD and post PhD students
• For the non biblio areas the indicator consists in the evaluation of
publications of each PhD or post PhD student, giving preference in the
order to books, to book chapters, to articles published in journals
belonging to the class A of ASN
•
The indicator is computed as the ratio between the weighted sum of the
publications of members of the Doctoral Body divided by three times the
number of PhD and post PhD students, and the weighted sum of the
publications of all members of all PhD and post PhD students of the same area
divided by three times their number
22
The evaluation of PhD programmes
• ANVUR will start the evaluation of PhD programmes by the end of
2016, with the evaluation of courses of XXIX, XXX and XXXI cycle
• The decision of not evaluating PhD programmes preceding the XXIX
cycle is due to the fact that the courses of XXIX cycle have been
deeply changed wrt the previous ones, through a large reduction in
the number of courses and a related transformation of the scientific
scope
• Moreover, the criteria and indicators for the accreditation and
evaluation have been discussed only starting with the courses of
XXIX cycle, so that it would not be fair to consider previous cycles
23
Timing of the evaluation
• With the exception of indicators IV6 e IV7, all indicators will be
evaluated for the three PhD cycles preceding the year of evaluation,
and the final indicators are averaged over the three years
• The sub-indicator 𝐼SB2 will take into account three consecutive cycles
the first of which concluded two years before the year of evaluation
• The sub-indicator 𝐼SB5 will take into account three consecutive cycles
the first of which concluded five years before the year of evaluation
• The indicator 𝐼V7 will be computed for the first time in 2017, when the
PhD programmes of XXIX cycle have been concluded, and will reach the
steady state in 2020, considering six consecutive cylces starting from
the XXIX
24
Timing of the evaluation
Ind. IV7
regime
I appl. Ind. IV7
valut. primo anno
Ind.
ISB2
Anno
2013
XXIX
ciclo
Anno
2014
XXX
ciclo
Anno
2015
XXXI
ciclo
Fine
2016
Fine
XXIX
ciclo
Fine
2017
Fine
XXX
ciclo
Anno
2016
XXXII
ciclo
Anno
2017
XXXIII
ciclo
Fine
2018
Fine
XXXI
ciclo
Anno
2019
Ind.
ISB5
Anno
2020
Anno
2021
Anno
2022
Anno
2018
XXXIV
ciclo
25
Summary
• The evaluation of PhD programmes
• Statistics on Italian PhD programmes
• VQR 2011-2014: The ANVUR perspective
• Introduction to ORCID
26
# PhD programmes per area
# PhD programmes per geographic area
XXX cycle PhD programmes: typology
area
prevalente
1
2
3
4
5
6
7
8a
8b
9
10
11a
11b
12
13
14
TOTALE
# dottorati
# dottorati
internazionali
# dottorati
industriali
in collaborazione
con atenei, enti e
imprese all'estero
43
52
46
19
84,5
113,5
46,5
31,5
29
109
84
51,33
21
68,33
74
24,33
897
0
3
0
0
4
1
1
1
0
7
2
0
3
1
3
1
27
1
1
2
0
2
3
3
1
1
9
0
0
2
1
0
0
26
2
4
4
0
6
3
2
3
4
11
9
2
4
6,5
3,5
1
65
XXX cycle PhD programmes: organisation
area
prevalente
# dottorati
1
2
3
4
5
6
7
8a
8b
9
10
11a
11b
12
13
14
TOTALE
43
52
46
19
84,5
113,5
46,5
31,5
29
109
84
51,33
21
68,33
74
24,33
897
# dottorati
convenzione
singola università
34
33
34
18
69,5
102
37,5
26,5
23
89,5
65
40,33
15
54,83
58,5
17,33
718
7
17
12
1
14
11,5
9
5
5
18,5
18
10
6
12,5
14,5
6
167
consorzio
consorzio e
convenzione
2
1
0
0
1
0
0
0
1
0
0
1
0
1
1
1
9
0
1
0
0
0
0
0
0
0
1
1
0
0
0
0
0
3
XXX cycle PhD programmes: # curricula
31
% distribution of members of Doctoral Body
XXX cycle PhD programmes: Distribution of R and X
Summary
• The evaluation of PhD programmes
• Statistics on Italian PhD programmes
• VQR 2011-2014: The ANVUR perspective
• Introduction to ORCID
34
La VQR 2011-2014
Il punto di vista dell’ANVUR
1.
2 prodotti per gli universitari, 3 per i ricercatori e gli affiliati degli enti
2.
5 classi di merito:
– Eccellente (peso 1), primo 10% della distribuzione della produzione
scientifica internazionale dell’area cui appartiene.
– Buono (peso 0,7), segmento 10-30% della distribuzione della produzione
scientifica internazionale dell’area cui appartiene.
– Discreto (peso 0,4), segmento 30-50% della distribuzione della produzione
scientifica dell’area cui appartiene.
– Accettabile (peso 0,1), segmento 50-80% della distribuzione della
produzione scientifica dell’area cui appartiene.
– Limitato (peso 0): segmento 80%-100% della distribuzione della
produzione scientifica dell’area cui appartiene.
– Non valutabile (peso 0): sono incluse in questa categoria anche le
pubblicazioni mancanti rispetto al numero atteso.
35
La VQR 2011-2014
Il punto di vista dell’ANVUR
3. Obbligo del commento da parte dei revisori
4. Definizione più precisa delle categorie di prodotti ammessi
(vedi scheda SUA-RD)
5. Data base dei brevetti e start-up resa disponibile da ANVUR
(vedi scheda SUA-RD)
36
La VQR 2011-2014
Il punto di vista dell’ANVUR
6. Possibilità per sottostrutture di grandi enti di ripresentare
lo stesso prodotto: dipartimenti per il CNR, sezioni per
l’INFN, osservatori per INAF
7. Migliore definizione degli affiliati accreditabili, e obbligo
dell’affiliazione all’ente per i prodotti assegnati all’ente da
parte degli affiliati
37
La VQR 2011-2014
Il punto di vista dell’ANVUR
8. Migliorare il livello di istruzione da fornire ai revisori e
rivedere le schede di revisione
38
La VQR 2011-2014
• Il decreto per la distribuzione del FFO 2014 destina al CINECA un
finanziamento per lo svolgimento della VQR 2011-2014
• Al tempo T il MIUR pubblica il DM sulla VQR 2011-2014 (31
gennaio 2015?)
• Al tempo T + 1 l’ANVUR pubblica il Bando e avvia una consultazione
della comunità scientifica (28 febbraio 2015?)
• Contestualmente, al tempo T + 1, l’ANVUR pubblica un Bando per le
candidature dei membri GEV che si chiude a T + 2 (31 marzo 2015?)
• Al tempo T + 3 l’ANVUR conclude la definizione dei GEV (30 aprile
2015?
39
La VQR 2011-2014
• Al tempo T + 2,5 si chiude la consultazione sul Bando (15
aprile 2015?)
• Al tempo T + 3 l’ANVUR pubblica il Bando definitivo (30 aprile
2015?)
• Al tempo T + 4 le Istituzioni concludono l’accreditamento degli
addetti alla ricerca (31 maggio 2015?)
• Al tempo T + 6 le Istituzioni concludono l’invio dei dati di
contesto (31 luglio 2015?)
40
La VQR 2011-2014
• Al tempo T + 5 i GEV pubblicano i criteri di valutazione (30
giugno 2015?)
• Al tempo T + 9 le Istituzioni concludono l’invio dei prodotti da
valutare (31 ottobre 2015?)
• Al tempo T + 17 i GEV concludono la valutazione dei prodotti
(31 giugno 2016?)
• Al tempo T + 18 l’ANVUR pubblica il rapporto finale (31 luglio
2016?)
41
Summary
• The evaluation of PhD programmes
• Statistics on Italian PhD programmes
• VQR 2011-2014: The ANVUR perspective
• Introduction to ORCID (Open Researcher and Contributor ID)
42
What do identifiers do exactly?
① Enable machine readability
② Disambiguate and enforce uniqueness
③ Enable linking and data integration
Persistent identifiers provide a
simple basis for digital data governance
43
Name ambiguity is a problem
•
•
•
•
•
Shared names
Different versions (full name vs. initials)
Transliteration
Name changes
Multiple family names
44
ORCID is a hub
APIs enable
connections
between:
•
•
•
•
works
researchers
organizations
other IDs
Repositories
Publishers
Funders
Other
person
identifiers
Higher
Education
and
Employers
Professional
Associations
45
ORCID is a registry
• Free, non-proprietary
registry of persistent
unique public identifiers
for researchers
• Supported by
member fees
46
Adoption and integration
ORCID issued the 1 millionth ORCID
iD during November 2014.
900.000
Creator
800.000
Website
700.000
Trusted Party
600.000
500.000
400.000
300.000
200.000
100.000
-
2012
2013
2014
47
Adoption and integration
• ORCID is developing fast in Europe, with consortia,
integrations and nationally coordinated projects and adoption
in Denmark, Portugal, Sweden, and the UK
• Finland, the Netherlands and Norway are exploring national
memberships and integrations
48
Connecting an ORCID iD
Journals are asking authors for their ORCID
iD at time of manuscript submission.
The authenticated iD becomes
a part of the paper
The author can pre-populate
form fields: preferred name,
affiliation, funding
Upon publication, the iD is
indexed by CrossRef, Scopus,
Web of Science, and other
services.
Information flows to ORCID
and linked platforms
49
Connecting an ORCID iD
Identifiers are being embedded in articles
DOI
50
ORCID iD as a link
• Over 130,000 articles have been submitted to CrossRef
with an associated ORCID iD
• These will start to flow into the ORCID registry before
the end of the year
• The same functionality is being developed by DataCite
for research datasets
• This “Round Trip” functionality is being implemented
now by CrossRef and DataCite
• Researchers who use their ORCID iD when they publish
will not need to manually update their record
• Systems linked to the ORCID registry can be
automatically updated
51
Benefits of membership
•
•
•
•
•
•
•
•
Systematic access to the registry using our API
Integration with institutional and national systems
Ability to read limited-access data in the registry
Ability to write to employees ORCID record – with permission
from the researcher
Automatic updating of systems connected to the registry
Improved data quality from other systems that integrate
ORCID iDs (e.g. Web of Science or SCOPUS)
Improved capture and sharing of data from other sources and
records
ORCID is international so global publications, collaborations
and activities can be linked to the researcher easily and visibly
52
How to join
Standard: One organization joins. Option for basic or premium benefit
levels. Discounts for non-profits and multi-year agreements.
Prices are based on the size of the institution and the level of access they
want - from $4,000 per year to $25,000 for a large organization with
premium membership
Consortium: One lead organization coordinates membership and
technical implementation with group participants. Fee based on number
of organizations in the group and benefit level.
There are a number of pricing levels for consortium membership – 90
institutions can join for $135,000 per year
http://orcid.org/about/membership
53
How could it work in Italy?
• Could be piloted in a group of institutions immediately
• Integrated into a new version of the national publications
database from launch
• ORCID is already integrated with repository and CRIS
(Common European Information Format) software, such as
Dspace-CRIS
• Once the local systems and the national systems both have
ORCID integrated, they would be able to interoperate and
share data easily and efficiently
• Using existing systems (such as CINECA IRIS) information could
be shared between universities as soon as they implement
the system
Once ORCID iDs are implemented in these systems, they would
be able to make better use of subscriptions to other resources,
and it would be a powerful incentive to Italian publishers to
integrate as well.
54
Adoption in Italy
There are 25,601 ORCID records with an Italian affiliation.
There are more than 500 universities, schools, departments and
research institutes linked to these records.
Italian visits to ORCID registry
50000
40000
30000
20000
10000
0
Jan-June 2013
June-Dec 2013
Jan-June 2014
Italian visits to ORCID registry
orcid.org
Il progetto ORCID ANVUR-CRUI
• ANVUR-CRUI diventano membro nazionale di ORCID
• Approfittando della VQR 2011-2014, ANVUR chiede a
tutti gli atenei di dotare ogni addetto alla ricerca di un
identificatore ORCID
• Tutti I prodotti sottoposti alla VQR sono associati agli
autori tramite ORCID
59
Il progetto ORCID ANVUR-CRUI
• A partire dalla prossima scheda SUA-RD, tutte le
pubblicazioni inserite sono associate agli autori tramite
ORCID
• Tutti i ricercatori (di ruolo?) italiani sono associati alla
loro produzione scientifica tramite ORCID
• L’obiettivo è di completare l’associazione pubblicazioniautori tramite ORCID entro la fine del 2016
60
Thank you
61