Between June ’14 and February ’15, SIGPLAN (the ACM Special Interest Group on Programming Languages) and AITO (the “Association Internationale pour les Technologies Objets”, a non-profit organization set up to support the ECOOP conference) conducted four surveys of their members and authors.
In 2013 SIGPLAN had started to explore the possibility of publishing some of its conference proceedings as issues of journals. That move was blocked by the ACM’s Publication Board. SIGPLAN was asked to demonstrate need and support from the community. At the same time, both SIGPLAN and AITO were asked by members to find a way to dispense with their respective paywalls (SIGPLAN publishes in the ACM Digital Library and AITO publishes with Springer).
The following four surveys garnered 1376 responses:
In terms of population the PLDI and POPL survey target attendees of the respective events (and co-located events) whilst the ECOOP survey target authors of submitted papers. All three survey mostly captured the feeling of members of the programming language community. The SIGPLAN survey, due to the viral nature of social media (twitter, G+, FB, LinkedIn) had a broader reach. Responses from members of other SIGs as well as unaffiliated researchers are included. A slight majority of all responses was from outside the USA.
SIGPLAN held a number of town hall meetings (POPL 2014, PLDI 2014, POPL 2015) in which members of our community strenuously argued against attempt to conflate journals and conferences. Their argument was that the battle had been won and that everyone outside CS understood that our conferences are just as prestigious as journals.
The results of the PLDI and the SIGPLAN survey debunk that myth: 60% of PLDI respondents feel that their employer requires journals publications. Textual comments indicate that journals are needed for promotion, tenure, and even attached to end-of-year bonuses. There is a 30% difference between the USA and the rest of world in this respect. Many US institutions (but not all!) have abolished the requirement to publish journal papers, but this is clearly not the case outside of that country.
The PLDI survey shows that authors who don’t have to do journals for career advancement, publish fewer journal papers. 77% of the SIGPLAN respondents believe conferences publish most of the important results in the field. 15% of the SIGPLAN respondents did not read a single journal paper in 2014.
Beside career advancement, the main reason people do publish in journals, according to the SIGPLAN poll, is for long form results that summarizes a completed research direction.
SIGPLAN’s position in this respect is to improve the conference reviewing process to better match the journal process and effectively alpha-convert our top “conferences” into “journals”. The hope is to relieve researchers from the pressure to produce artificial journal papers to satisfy administrative requirements without discouraging authors who truly have a long form result from publishing it. The current proposal is to have TOPLAS publish both conference versions of papers and suitably extended long form papers.
The community seems to supportive. 74% of SIGPLAN respondents are in favor. 70% of PLDI respondents are in favor. 79% of POPL respondents are in favor.
Another approach has been put forward, namely to ensure that conferences are properly index by major bibliometric databases. In the long run this will reduce the bias against CS publication as their impact factor will be fully taken into account. Nothing prevents the community from pursuing both paths as they are not mutually exclusive.
Open access is a complex issue as it involves long term sustainability questions. The issue is further muddled by the attempts from publishers to confuse authors with a number of seemingly appealing choices. But in any conversation with our member the meaning of open access is clear, for most of them open access means no paywalls between papers and readers.
If we get rid of paywalls, the question becomes how to fund the storage of the research results. SIGPLAN and AITO have tried to elicit opinions from their respective communities about how to replace the existing financial model based on publishers charging readers.
There is strong agreement across the board about the desirability of OA in the abstract: 81% of POPL respondents find OA important, 85% of PLDI respondents want OA for their papers, 73% of SIGPLAN call OA a moral imperative.
The question of how to fund OA divides SIGPLAN respondents: 28% are in favor of author processing charges while 41% argue for increased conference registrations, the remainder want us to find other means. In the case an author processing charge was selected, how much would be a reasonable price (in USD) for a paper? 63% of respondents are willing to pay no more than $100 per paper. In the POPL survey, 83% chose $100 or less.
The ECOOP survey was more concrete, authors of submitted papers were asked if they would support publishing in OA with Dagstuhl’s LIPIcs in 2016, 84% were in favor, a switch in 2015 gained 73% support. AITO decided to switch in 2015. There will be no additional cost to authors as the 15 Euros paper processing charge can easily be covered by registration costs. When asked about a similar move away from the ACM to an unspecified OA publisher by 2017, 70% of SIGPLAN respondents were in favor.
The textual comments in the SIGPLAN survey suggested little support for ACM’s current operating model. Multiple respondents mentioned arXiv and LIPIcs as alternatives.
Reading the SIGPLAN text comments suggest a disconnect between what respondents expect from the ACM and what the Association views as its own role. This is reflected by the 74% of SIGPLAN respondents in favor of scaling down ACM expenses, include the so-called Good Works, to fund OA.
Many respondent seem unaware of many of the activities of the ACM, instead they repeatedly emphasize the core missions of organizing scientific meetings, making scientific results broadly available and setting and mainting standards of quality and ethics. The comment suggest that there is some unhappiness about how the association fulfills the first two of those missions.
In addition, the ACM’s DL is widely criticized as being inefficient, unwieldy, overpriced, and generally subpar. The respondents value the DL’s meta data more than anything else it provides.
Certainly communication and transparency could be improved. Some repsondent asked what the “Good Works” are, other were surprised by the use of funds outlined in introduction to the SIGPLAN survey. But a more profound realignment of the ACM’s activities may be worth considering.
Please send comments to Jan Vitek. Boston, March 29th, 2015. (v125)