New Paper Categories and Peer Review Criteria

Submissions for the Japanese Society for Artificial Intelligence have been classified into four categories: Papers of AI Frontiers, Original Papers, Exploratory Research Papers, and Short Papers. The category of Papers of AI Frontiers was newly created in May 2003 to develop new areas of study and to vitalize JSAI. Unfortunately, however, few submissions have been made to the category of Papers of AI Frontiers.

Having reviewed the situation, the Editorial Committee held a series of discussions, including some on what attractive papers would be and what the peer review criteria should be. In September 2010, the Committee decided to dissolve the category of Paper of AI Frontiers constructively and to create Concept Papers (papers deemed attractive and developmental, applying new concepts or new methods based on novel ideas) and Papers on Practical AI Systems (papers on various knowledge and know-how of practical systems related to artificial intelligence) in the category of Original Papers.

We summarize the categories of papers that JSAI accepts and the peer review criteria below, including conventional papers. Please look at it. We look forward to your active submission to each category of papers. The period from September 2010 to December 2010 is the transition period. We will accept papers submitted to the previous categories.

(1) Categories of submissions
Table 1 presents the categories of submissions.

Table 1 Categories of submissions

* The number of pages of a paper includes the title, figures and tables, About the Author, and all other contents.

(2) Peer review criteria for submissions
Peer review criteria for each paper category are presented below.

(a) Original Papers (Concept Papers)
When a field of study has been established to a certain extent and is becoming mature, papers with a high degree of perfection including rigorous formulation and verification of usefulness by reliable evaluation experiments become necessary. This can be a natural developmental tendency of science. However, it can also be a situation that is unsuitable for the cutting-edge nature that artificial intelligence research has today. State-of-the-art research that has opened up the field of artificial intelligence research cannot be judged using criteria for papers in the matured fields described above.

For example, many pieces of research, such as papers of Minsky’s Frame Theory, general frame problems, and the proposal of RoboCup, are achievements that have contributed greatly to the progress of science and technology. However, at the point of publication, all those research efforts were difficult to evaluate using the criteria in the matured fields of study. In addition, such research is a proposal of concepts or methods, or presentation of problems that constitute “a meaningful proposal” even the verification is insufficient at the time of the proposal. Although great efforts have been made for such proposals, it might take some time for its usefulness to be readily apparent.

Concept Papers are intended to accept such papers. The peer review policy does not place much emphasis on quantitative evaluation or objective evaluation. Determination shall be made under the responsible of members of the Editorial Committee.

(b) Original Papers (Technical Papers)
Technical Papers shall include novelty and usefulness in the content, have credibility because of being fully substantiated, and be deemed useful for members. The evaluation criteria are the same as they were before.

(c) Original Papers (Papers on Practical AI Systems)
Major examples of Papers on Practical AI Systems are shown below, but are not necessarily limited to the following.

1. An effective mode of using existing technologies according to a problem
2. A useful mode of combining existing technologies
3. Development of a useful tool
4. Development of interesting application
5. An effective mode of realizing existing technologies
6. Analysis of environments in which AI systems work effectively
7. Evaluation of business values and developmental costs, including technical content
8. Systematization of know-how in the development and operation of AI systems
9. Other novelty matched to the main point of this special feature

In traditional criteria for peer review of papers, objective evaluation based on experimentally obtained results is necessary from perspectives of novelty, usefulness, and reliability. However, in practical AI systems, it is often difficult to satisfy the requirements perfectly. Therefore, as the peer review policy, if the following requirements are satisfied, then there shall be no difficulty in employing qualitative evaluation: the paper includes discussions that members can develop for application to other cases; JSAI determines whether there is a meaning to give authorization; there is a justifiable reason that the author is unable to present sufficiently objective evaluation results; and actions and effects obtained from the mechanism to be proposed are argued in a logically consistent manner.

(d) Exploratory Research Papers
Exploratory Research Papers should contain ideas that promise future development and certain research results, should have clearly distinguishable achieved parts and incomplete parts, and should be deemed to bring useful findings to members.

(e) Short Papers
Short Papers should have novelty in contents and should be deemed useful for members. Full substantiation is not required.