OCSP - Online Calculator for Scientific Performance Indonesian Institute of Sciences

This page provides a brief explanation about the SFPM (Scientific and Financial Performance Measure) Model and its application [1]. You can also study the detail through the full-text version available at arXiv:physics/0508052.

1. Introduction

   Management is an important aspect of human-being and its activities, including any kind of scientific activities. Management as the regulator and also the executor of regulation needs appropriate tools to implement the existing regulation. Regulation has been created as a common norm and commitment which should be followed by all (individual and institutional) fellows in order to establish a mutual relationship. Mutual relationship is a crucial point to realize every fellow's purposes and motivate all of them to work together.

   In the context of scientific activity, scientific management plays an important role, much more than the other non-scientific (bureaucracy, business, etc) activities. This reflects the nature of scientific activity which is strongly based on the individual with unlimited degree of freedom. Because independency and freedom are necessary conditions for any scientific activities. On the other hand since a scientific activity is mostly supported by public fund, all scientific fellows and its activities should be accountable for public. In a real life, however fully transparent, easily-understandable and accountable scientific activities are often difficult to be realized. This is moreless due to the nature of scientific activity which is in most cases invisible and unpredictable. Fortunately, in contrast with another non-scientific activities, scientific activity is always supposed to generate an objective and calculable outcomes in a period of duration.

   This means there is an urgent need for a specific tool to measure scientific performance based on the scientific outcomes. Because scientific outcome is the only part which is measureable. Moreover, performance measurement is neither limited to scientific performance nor long term economic potential. So far, scientific activities are almost justified by the aspect of economic potential using the method of cost-benefit analysis [2]. While the scientific performance is measured naively based on the scientific outcomes. This method clearly separates scientific and financial aspects, although both aspects are closely related each other. More than that, in daily practice it is hard to measure the objectivity on cost-benefit method since it uses absurd references as future potential which is yet unpredictable.

   It is also known a more complete method called as the scientific and technical human capital (STHC) [3] which observes a lot of aspects. Though the method looks ideal, this model is very complicated and involves a lot of subjective parameters. This reduces the accuracy and validity of the method as an easy, transparent and obejective measurement tool.

2. Theoretical framework

   According the introduction in the preceding part, it is clear that as long as concerning scientific activity, performance should be measured based on the achievement of scientific outcome without considering its process. Inversely, this point makes scientific activity is easy to measured and quantized. Motivated by this fact, we propose an alternative tool called as OCSP Model [4].

   As mentioned above, the first assumption in the model is all measurements are based only on scientific outcomes regardless its process. The scientific outcome itself is defined as : all outcomes generated in a scientific activity which have been approved by independent third party(ies) in a form of either scientific documents or other real activities.

   The second assumption, the measurement is done in a year basis and takes into account all outcomes in the last fiscal year.

   Next assumption is, each outcome is ranked based on its difficulty to achieve, and is provided by an appropriate point based on its scientific content. The order number (N_O) of all relevant scientific outcomes must be in order and not doubled. On the other hand, its scientific point (S_P) may be the same with another neighboring scientific outcomes, but it must be smaller (greater) than another scientific outcomes with different points above (below). Determination of the order and the point of scientific outcomes may differ depend on the nature of each field of science.

   Further, it is also assumed some parameters of maximum scientific point (P_M), descending rate of scientific point (P_D, in percent) and total scientific point treshold per-scientist (P_T). These parameters are the same of all fields of science. This method enables universal evaluation and comparation among different branches of science. It should be remarked that the absolute value of scientific point itself is less important, because it only represents the scale of differences among scientific outcomes. Once the maximum scientific point and its descending rate have been determined, a series of available scientific points for each scientific outcome can be obtained as :
   

   SP = { PM , PD x PM , (PD)2 x PM , ... , (PD)nO-1 x PM , (PD)nO x PM } ,

   (1)

   
   where n_O denotes the number of relevant scientific outcomes in a field. Concerning the appropriateness, it is natural to restrict the values for following parameters :
   

   minimum SP > 1 ,	(2)
   PM , PT > 100 ,	(3)
   90 < PD < 50 (percent).	(4)

   Economic aspect related to financial performance is represented by the economic coefficient (C_E) which holds for all branches of science. This parameter should be determined initially, and thereafter can be made varying automatically depending on the macro economic indicators (inflation, economic growth, currency rate, etc).

   Finally, using all assumptions above the ratio of Scientific Performance (SP) and Financial Performance (FP) are formulated as the following equations. From these equations, it is clear that FP is related directly to the scientific point and order number of each scientific outcome.

   (5)

3. Application

   Regarding the research management, in the evaluation based on OCSP Model a regulation in two levels of management is required to determine all parameters introduced above. That is :

   • Official bureaucracy with authority on scientific activity.
     The management in this level is responsible for determining the values of global parameters which hold for all branches of scientific activities, i.e. P_M, P_D, P_T and C_E.
   • Related scientific community.
     The scientific community in a specific field is responsible for determining the order and the value of scientific point of all relevant scientific outcomes. Because these parameters are unique for a specific field.

   Soon after determining all parameters. the system is ready for evaluating previous scientific activities and also decision-making of future scientific activities as well. As an example one can use an online facility available on the net [5].

4. Conclusion and discussion

   Using the SFPM Model, a simultaneous relation between scientific and financial performances has been formulated based only on the scientific outcomes. Several advantages of measurement tool based on the SFPM Model are :

   • Since it is based only on the scientific outcomes, the obejectivity and transparency of measurement can be guaranteed.
   • A single year base method makes the evaluation process and decision-making for next fiscal year easier, since the result reflects an up-to-date real condition.
   • In long term, all annual evaluations of each fellow can be compiled to implement a compensation system. Example : distribution and allocation of human resources and funding to each individual or institution.
   • The same tool can be used to evaluate incoming proposals regarding the promising targets claimed in.
   • The result of financial convertion can be used as a standard to decide the budget approved for each individual or scientific project.

   Implementing the OCSP measurement tool consistently and continously in long term for each individual, scientific activity and institution could provide performance indicators in each level. This has a great potential to help identifying the fundamental problems in regional or national scale. Combining the result with a strict compensation system could also improve the whole scientific performance.

   The parameters introduced above are not absolute and only indicate the scale, which make them to be determined easily without requiring deep consideration. However, more serious consideration is still required for few of them, which especially constitute all fields absolutely as descending rate of scientific point (P_D) and total scientific point treshold per-scientist (P_T). The parameter P_T can be determined in a comprehensive way by for instance taking the mean of all scientific outcomes generated by scientists around the world. The data can be retrieved easily through the existing databases available on the net. This result further can be used to determine the appropriate mean of parameter P_D. Therefore, further research utilizing available databases providing global and local scientific outcomes on the net is very recommended.

References :
1. L.T. Handoko, Proceeding of ASEAN Science and Technology Week - Sub-Committee of Science and Technology Infrastructure and Resources Development, Jakarta (2005); arXiv:physics/0508059.
2. B. Bozeman and J. Melkers, Evaluating R&D impacts: Methods and practice, Boston, Kluwer Academic Publishers (1993);
   R. Kostoff, H. Averch and D. Chubin, Evaluation Review 18 (1994) 3.
3. G. Becker, Journal of Political Economics 70 (1962) S9;
   T. W. Schultz, The economic value of education, New York, Columbia University Press (1963);
   M. Polanyi, The tacit dimension, London, Cox & Wyman (1967);
   M. Polanyi, The logic of tacit inference : Knowing and being, London, Routledge & Kegan Paul (1969);
   B. Bozeman, Policy Studies Journal 14 (1986) 519;
   P. Bourdieu, Handbook of theory and research for the sociology of education New York, Greenwood (1986);
   P. Bourdieu and L. Wacquant, An invitation to reflexive sociology, Chicago, University of Chicago Press (1992);
   J. C. Coleman, American Journal of Sociology 94 (1988) S95;
   J. C. Coleman, Foundations of social theory, Cambridge, The Belknap Press of Harvard University Press (1990).
4. L.T. Handoko, Journal of Engineering and Technology Management (2005) under submission.
5. OCSP - Online Calculator for Scientific Performance, http://www.koki.lipi.go.id (2005).

DISCLAIMER : This system is dedicated only for scientific simulation purpose. LIPI has no responsiblity on all contents and any aspects caused by the contents available in this site.