Skychain March 2021 project development update

Preparation

The test was held in February with the support of Department of Pathology of Moscow Central Hospital, which has been our partner since our very start. During the test we have compared the results of diagnostics of three participants:

  • the pathologist of Moscow Central Hospital with 1 year of medical experience;
  • the pathologist of Moscow Central Hospital with 5 year of medical experience;
  • Skychain prostate cancer neural network.
  • AT (atrophy);
  • O (acinar carcinoma)
  • PENST (foam-cell carcinoma);
  • PROT (ductal carcinoma);
  • VHR (chronic inflammation);
  • N (normal tissue);
  • PIN (prostatic intraepithelial neoplasia).

Results

We have recieved the results of each participant on each exact slide.

Explanation: Cohen’s kappa measures agreement between two evaluators, each classifying N elements in C mutually exclusive categories. In our case, the Cohen’s kappa will be calculated on 108 test slides for N experts + the prediction of our neural network, that is, we will get N + 1 values.

Cohen’s Kappa Formula:

where pо is the relative observable agreement between the evaluators (identical in accuracy), and pе is the hypothetical probability of a random agreement, using the observed data to calculate the likelihood of each observer randomly seeing each category.

Simply put, Cohen’s Kappa shows how the opinion of one expert coinscides with the opinion of reference standard. The closer it is to 1, the more the opinions coincide.

X-axis — doctors making a diagnosis + neural network, Y-axis — Cohen’s kappa. This diagram shows the distribution of Cohen’s kappa for 2 doctors and the neural network. The legend on the bottom left in color indicates the belonging to each of the classes. The graph displays the minimum / maximum value. The body of the candlestick is calculated as the median value + — the standard deviation.
X-axis — doctors making a diagnosis + neural network, Y-axis — Cohen’s kappa. The different colors correspond to different doctors for clarity.

In general, we can conclude that Skychain has outperformed both experts in several classes and showed quite comparable results in others.

We were quite surprised to see the results of our work being competitive with real doctors. However, there is still much to do, since we are able to make mistakes in diagnosing several classes. We plan to use more data on the “weak” categories to show better results in our next test.

But what about the time?

Of course, as speed is one of the biggest advantages of AI, Skychain managed to do its analysis much faster than both experts did.

  • Pathologist with 1 year of practice — ~5,5 hours
  • Pathologist with 5 years of practice — ~4,7 hours
  • Skychain — ~0,75 hours

Curious case

There was also one curious slide we would like to tell you more about.

Tissue sample. As you can see, it has some few blue areas, which mean that Skychain neural network sees it as cancer.
  • Pathologist with 1 year of practice
  • Pathologist with 5 years of practice
  • Skychain
  • Reference standard

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Skychain Official Channel

Skychain Official Channel

Blockchain infrastructure aimed to host, train and use artificial intelligence (AI) in healthcare. Our website: https://skychain.global/