How Accurate Is the Cyprus Tomb Register?

The register, introduced in The Cyprus tomb register, holds 3,837 tombs, and for 2,324 of them we have read the excavation report entry by entry. We have measured how often those readings are wrong, in six different ways.

The short answer is that most of what the register lacks was never in the reports to begin with. Our measured error rate comes to a few per cent on most fields.

What the reports leave out, and what we get wrong

A tomb record can fall short in two ways. The report may say nothing (no word on looting, no height for a jug), and then neither does our record. Or the report says something and our record does not match it. That is our error, and only that is ours to correct, so we count the two separately.

The reports say nothing about looting for 89% of tombs, give no count of the people buried for 76%, and give no find-spot for 60% of their entries; since these shares are counted on our records, they also take in the few values our reading missed. On those fields our error, measured against the reference tombs described below, runs from 6% to 11%. Reports almost always say how many objects an entry holds and what they are made of, and there our error is a few per cent at most: 3.7% for the count and 1.9% for material on the reference tombs.

Our own error is largest on the date of the tomb. One reference tomb in seven has a different date range in the register from the one in the reference catalogue, mostly where a printed label such as "MC III/LC IA" or "end of the Cypro-Geometric III" had to be fitted to our broader periods. The number of individuals is wrong about one time in ten, and the record of looting a little more often. These figures rest on small samples, of 36, 61 and 53 cases.

Of the tombs dug between 1905 and 1934, most of them by the Swedish Cyprus Expedition, 80% have a full catalogue in the register. For tombs dug before 1905, most of them excavated by the British Museum in the 1890s and published as summary lists, the share is 4%. For tombs dug since 1974 it is 17%, because for most of them we so far hold only the short annual notices.

The reference tombs

We prepared a reference catalogue for 56 tombs, drawn from every kind of report in the register: the Swedish volumes, Dikaios's Vounous, the British Museum's notebook, the annual excavation notices of the Bulletin de correspondance hellénique, monographs and articles. Each reference tomb was catalogued twice, independently and with particular care, and the two versions reconciled. Forty of the tombs were chosen to cover every kind of catalogue layout we had met, and our recording practice was worked out on them. The other sixteen were drawn at random and set aside, so that they test the register on tombs that played no part in shaping it.

Across all 56 tombs, the register contains 97.6% of the 2,479 entries in the reference catalogue, and 93.8% of the register's entries for those tombs correspond to a real entry in the report. In the sixteen test tombs it contains all 406 entries, and 98.9% of its entries are real. Material, ware, shape and dimensions are right for 94% to 98% of entries, and burials are recorded nine times in ten. The weakest point is the find mentioned only in passing, without a catalogue number: the register includes just 58.5% of these, which are kept apart from the catalogue and never counted among a tomb's contents. An earlier version of the register, of 24 October, scored a little higher on the 56 tombs (99.3% of entries present, 96.4% real). A handful of tombs lost ground in the latest revision and are being corrected; the sixteen test tombs did not change.

The reference catalogue also tests for the worst way of confusing silence with fact, which is to turn it into a negative, such as "not looted" for a tomb whose report never mentions looting. Of 123 facts on which the reports are silent, the register now records 4 as none or zero, where the version of 24 October recorded none of them so; in the sixteen test tombs it does so for 0 of 33.

A second, independent reading of every catalogue

The reference tombs are a small sample, catalogued slowly. The second check covers everything. Every page of catalogue in the register, 3,552 pages in all, was read a second time without sight of our records, and the two readings were then compared entry by entry, 29,170 entries in all.

Agreement between two readings is measured with Cohen's kappa, which discounts the matches two readers would make by chance and is usually taken as near-perfect above 0.8. Material scores 0.92, ware 0.91, object type 0.91 and shape 0.89; an object's stated origin is weaker, at 0.76.

Agreement leaves open which reading is right. Every disagreement, 11,701 questions in all, was settled by going back to the publication, without knowing which reading was ours, and each decision records the words in the text that settle it. From those decisions we estimate the register's error at 0.9% for material, 2.5% for ware, 1.5% for shape, 0.7% for the count and 2.5% for dimensions. This is the check that found the Vounous metres.

The decisions were then checked in turn. Thirty of them were taken at random and examined again against the publication, without sight of the original verdicts, and 24 were upheld. Among the six that were wrong, one had read "glass-like material" as glass, and two of the three corrected counts were mistaken: on the page, "57 + 58. Silver pin" is one object, and "ten fragments" of one amphora are also one. Six mistakes in thirty mean that the error rates above are uncertain by about a fifth of their own size, so that a ware error of 2.5% could be 2% or 3%.

Keswani's tomb-by-tomb counts

Every check described so far compares our readings with one another or with the publications themselves. One comparison is with an independent count made by another scholar from the same books. For Mortuary Ritual and Society in Bronze Age Cyprus (2004), Priscilla Keswani counted pots and copper objects tomb by tomb from the excavation reports. All 40 of her Swedish chambers at Lapithos are in the register. Our pot counts agree with hers exactly in 32 of them and our copper counts in 30, and the correlation between the two (where 1 would mean that they rise and fall in perfect step) is 0.997 for pots and 0.998 for copper [6, Tables 4.11a–c, pp. 206–213]. At Vounous the copper counts match in 27 of 28 Dikaios tombs and the pots correlate at 0.99 [6, Tables 4.7a–c, pp. 197–203]; at Enkomi, across 17 Swedish tombs, the pots correlate at 0.9998 [6, Tables 5.9a–d, pp. 231–239].

The pages left unread

A tomb can also be missing altogether, because it is mentioned only in passing in a book that we did not read through for the register. To test how many such tombs there might be, we drew 5,000 pages at random from works on Cyprus that we had not read for the register, and read them. They held no new tomb and no new fact about a known one; the only Cypriot tombs they mentioned were two that the register already had. That does not rule out missing tombs, but with 95% confidence the books we have not read through hold fewer than about one new tomb per thousand pages.

Two accounts of the same tomb

The Swedish Cyprus Expedition left us a natural experiment. Its excavators kept field diaries in Swedish, and the expedition later printed its catalogue in English [7; 8]. The pencil page at the head of this article comes from one of those diaries: its entry for Tomb 15 at Amathus, written in 1930. The register holds 6,497 entries from 11 diaries, covering 120 tombs, and 5,624 of them match one printed entry each. If our reading of either kind of document were careless, the two would often disagree. They agree on the number of objects 98.9% of the time and on the material 99.4% of the time; heights agree to within 10% in 97.8% of cases and to within 5 mm in 78%.

Where they part company is in the names. The shape family differs for about one object in nine (a dish or a kylix in the diary becomes a bowl in print, and a flask becomes a bottle), and the ware family for about a fifth of the vessels. These differences are the expedition's own: the field labels were provisional, and classification was work done for the publication. The register takes the printed value, and the diaries add what the books leave out, including 570 links between objects and burials.

Some tombs have been published more than once, and for 380 of them the register holds two or more publications. Where at least two give a value, our records of them differ on the type of tomb in 17% of 190 tombs, on the number of people buried in 19% of 53, and on ancient looting in 9% of 22. They differ on the date in half of 125 tombs. Where both dates can be placed on our scale of periods, one range lies inside the other in 43 of 61 tombs, which means that the publications differ in how closely they date the tomb or in how long they think it was in use. In 14 tombs the two ranges overlap, and in only 4 do they not overlap at all. The tomb pages show both readings on one line, and our analyses use the primary report.

The systematic errors found so far

Errors that scatter at random blur a pattern. The ones to fear lean the same way across a whole report, usually because the reading missed one of the report's conventions, as happened with Dikaios's metres.

C. F. A. Schaeffer dug at Vounous in 1933 and catalogued his tombs in French, and entry after entry in his inventory reads "t. c. rouge lustré": terre cuite rouge lustré, red lustrous terracotta [2, Appendix II]. In 98 entries, holding 129 objects, the register recorded those words as Red Lustrous Wheelmade, a Late Bronze Age ware. Schaeffer meant Red Polished, and he says so in the same book. Describing the pottery of the prehistoric village the Swedes had dug at Alonia ton Plakon, west of Lapithos, he writes of "une poterie couverte d'un engobe rouge-lustré, red polished ware" [2, p. 8]. We found the mistake while preparing this article. The 98 entries are being moved to Red Polished, where Schaeffer's own usage puts them.

Many reports run one tomb's catalogue straight into the next, so that a new tomb can begin halfway down a page. About 800 entries stand on such shared pages and were assigned to different tombs by the two independent readings. Twelve entries filed under Vounous Tomb 32A turned out to be the list for Tomb 32. Totals per tomb in reports laid out this way carry that uncertainty, while counts of a class of object across a whole cemetery are unaffected.

Sometimes the reading is faithful and the page itself is doubtful. The British Museum's notebook of its 1896 excavation at Enkomi, published by Veronica Tatton-Brown in 2003, says of Tomb 76: "Deep tomb near 45 & very like it. Full of fallen in rock & water about 120ft from surface. 3ft of water" [3, p. 42]. The register records a depth of 36.6 metres. The next deepest tomb in the register is another British Museum tomb measured in feet, Kourion Tomb 104, "Like a well, about 70 ft. deep" [4, p. 81], recorded as 21.3 metres, and after those two none is deeper than about seven metres. We record what the source says. What the register still lacks is a flag for values that look doubtful, so that an analysis of depths does not take 120 feet at face value.

The checks make mistakes of their own. In J. L. Benson's publication of the Pennsylvania excavations at Kourion-Bamboula, each tomb's list gives an object's number, what it is and the depth at which it was found: "B 443. LC III Decorated bowl; 70 cm." [5, Tomb 3]. The second reading took some of these depths for heights, and in 17 cases the arbitration sided with it. The register had them right, which is why no dimension proposed by the second reading entered the register without a further check.

How much error the results can take

The register has already been used in a study whose tests were set down in advance, and its headline result concerns iron. In 40 tombs from Late Cypriot III to the Cypro-Classical period, iron came in through blades, while pins, rings and brooches stayed bronze until the Cypro-Classical. Within the same tombs, the odds that a blade or point is iron are about 75 times the odds that an ornament is.

To see how much error that result can absorb, we introduced errors at random, relabelling iron as bronze and blades as ornaments (and the reverse) at rising rates, and repeated the test 400 times at each rate. At about 1%, close to the error measured for material, the odds ratio, which is 74.5 in the data as read, comes out at 53, with 95% of trials falling between 27 and 92. At 10% it is 7.4 (4.0–14.2), and at 20% it is still 2.9 (1.6–5.2). Only at about 30% does the band of trials reach 1, the value that would mean no difference at all.

Random error of this kind can only weaken a pattern, pushing the result towards no difference. An error that leans one way can also make one. To bring the iron result down to nothing by that route, about 85 of the 99 iron blades in the sample (86%) would have to be bronze ones misread as iron. The other strong result is Keswani's claim, now tested tomb by tomb, that burial posture changed during the Bronze Age: among 129 burials, 72% of the Late Cypriot dead lie extended, against 20% of the Early and Middle Cypriot dead. With postures misread at random at the measured rate of 3.7%, the gap of 52 points stays at 49 (between 42 and 54 in 95% of trials), and at 30% it is still 21 (between 5 and 36).

So the danger lies in the systematic errors, which are also the hardest to see. A second reading cannot catch an error that both readings make, and the reference tombs catch it only within their own sample. Schaeffer's inventory is the clearest case so far: one French phrase read the same wrong way in 98 entries, putting a Late Bronze Age ware into tombs where its excavator recorded Red Polished.

What has not been measured yet

Some parts of the register have no measured error at all. Goods are linked to a particular burial for fewer than a fifth of the burials described, and those links have not been scored. The type numbers that the reports print, like the "Type I b" Dikaios gave the snake jug, are recorded for 2,387 entries and have been checked in only 30 cases, of which 21 were right. The links between catalogue entries and the museum records of the same objects have no reference set to score them against. If you find a record that does not match its page, please tell us through the contact page, giving the tomb and the page of the publication.

On 23 October we drew 48 tombs at random for an audit and set them aside; none of the checks described here has used them. They will be catalogued afresh and compared with the register once, when this first edition is complete, with nothing adjusted after the result is known. Until those 48 tombs are scored, every figure above is provisional.