Well, the news of Maryland's Integrated Ballistics Identification System database being a failure has made the rounds of the blogosphere. Kim commented on Wedneday, so did Say Uncle (with an Instalanche). Triggerfinger, Keith Devens, and No Quarters, did too. The Geek with a .45 gives a link to the actual report in a PDF file (graphic, rather than text file, though,) and Irons in the Fire commented on that.
I've been pretty busy, but I had a chance to read the report yesterday, and it's an interesting expansion on the other reports I've read. There are two from California's ballistic imaging feasibility study, and the original Maryland study. All of these reports reference New York's system, but I have yet to find a study of that system specifically.
The general consensus of all of the blog pieces was a sarcastic "big freaking surprise!" which is understandable given our stated biases. The response from the gun
I am fairly green, could you explain why the idea would not work.Several respondents made a valiant effort to explain the problems inherent in the system, but a couple of paragraphs is insufficient. Like most controversial topics, there's a whole lot of "there" there, and no simple two- or even ten-sentence response is enough. Sometimes I forget that a lot of people don't have the basic information I've accumulated over the last ten years. (Generally not, though, which is one reason my posts - like this one - tend to the Den Bestian in length.)
I can see their problem of the guns not being indexed, but would it would seem that that could be solved by indexing all the guns.
So here, in some detail, is a dissertation on just some of the problems with the concept of "ballistic fingerprinting" as a crime-fighting tool.
First, let's see what the gun
When a gun is fired, identifying marks are made on the bullets and cartridge casings. Those marks, called ballistic fingerprints, are as unique as human fingerprints - no two firearms leave the same marks. The marks are also reproducible - every time a gun is fired it leaves identical marks. The uniqueness and reproducible qualities of ballistic fingerprints can provide a critical tool to law enforcement for solving gun crimes by rapidly identifying the specific weapon that was used in a crime.That's the basic idea they're selling. The VPC says, surprisingly, nothing that I can find concerning ballistic fingerprinting - possibly because they outspokenly agitate for a handgun ban and may see ballistic fingerprinting as a step backward. The Coalition to Stop Gun Violence has a web page on the topic, and a full report on the wonders of ballistic fingerprinting. Interestingly, both the Brady Campaign and the CSGV hang a lot of weight on how ballistic fingerprinting could have aided in the capture of Muhammed and Malvo in the D.C. Sniper killings. The CSGV says this:
In October 2002, the world watched helplessly as the Washington, D.C., area experienced a sudden rash of gun violence. Seven shootings, six of which were fatal, occurred in just two days in various D.C.-area suburbs. Investigators recovered bullets from some of the shootings and examined them under a microscope. Unique, microscopic markings indicated that the bullets had been fired from the same gun. A sniper was on the loose.The Brady Center says much the same:
--
In retrospect, the microscopic markings on the bullets that police recovered early in the investigation provided strong evidence about the killers’ identities. Those markings, and similar markings left on cartridge cases that were recovered later, constituted a unique “ballistic fingerprint” of the specific gun the snipers used. If police could have identified the make, model and serial number of the snipers’ gun from its ballistic fingerprint, they could have used that information to access the existing crime gun trace system. That, in turn, would have led them directly to the Tacoma, Wash., gun store where the snipers had acquired the Bushmaster XM15 assault rifle they used to murder 10 people in the D.C. region while terrorizing millions more.Unfortunately, because there is no existing, comprehensive system linking ballistic fingerprints to the guns that produced them, the evidence police recovered so quickly could not help them identify the killers.
In the Maryland sniper shootings, police rapidly matched bullet fragments from each victim to prove that the same gun was used in all of the shootings. The technology to match bullets to firearms is known as "ballistic fingerprinting." It worked and provided police with important crime leads. But what was missing, what police desperately needed, was a nationwide database of the ballistic fingerprint of every gun before it is sold so that police could determine not just that the bullets came from the same gun, but which specific gun - manufacturer, model, serial number - the bullets were fired from. That would have helped police trace the sniper after the very first victim.Of course, it's all the NRA's fault that there's no such database:
Because of opposition from the gun lobby and the National Rifle Association (NRA), efforts to expand ballistic fingerprinting to include all new guns have been blocked in Congress and state legislatures. Ballistic fingerprinting technology is proven and reliable. What is lacking is the political will for politicians to stand up to the gun lobby and establish a comprehensive ballistics database that would help law enforcement solve more gun crimes and catch more gun criminals.Here's the problem, though. What they say (and this is overwhelmingly true for these groups) is only partly (in this case, minimally) true. There's a whole lot of information they neglect, gloss over, bury, and avoid.
Let's look at what the current ballistic fingerprint (I'm going to abbreviate it BF from now on) databases are attempting. In Maryland's and New York's, all new handguns sold in these states must be provided from the manufacturer with one or two fired cartridge cases. When the gun is sold, those cases are forwarded to the proper ballistics lab along with the serial number, make and model of the handgun. Supposedly no direct information about the purchaser is included, but you can imagine how long that would actually last. The NRA's primary objection to BF is that it's backdoor gun registration.
Anyway, the cartridge cases are then entered into the BF database by being digitally imaged by a trained ballistics technician, and those images are linked to the make, model, caliber and serial number of the firearm. Then, if any spent casings are found at a crime scene, that recorded information will be available for comparison with the evidence collected. The BF database will be searched by a sophisticated program looking for a match. What does the software look for? Well, as the Brady Center explained, it looks for those "unique markings" - markings "as unique as human fingerprints."
The only problem is, they're not.
The first feasibility study done for the California legislature explained:
As bullets and cartridge cases are expelled from a firearm, microscopic markings are left on the bullets and the cartridge cases from the firing pin, ejector, barrel and other internal mechanisms of the firearm. These marks are unique to each firearm and are substantially reproduced each time the firearm is fired. The size, shape and location of these marks can be used to establish a smaller pool of firearms that share characteristics for comparison purposes. The individual nature of these marks can be used to conclusively identify a specific firearm as having fired a particular bullet or cartridge case. Ballistics imaging is often referred to as “ballistic fingerprinting” or “ballistic DNA.” Unlike DNA, which cannot be altered, some markings made by firearms may change over time with normal wear and tear of a firearm. Some preliminary studies suggest that some firearms marks may change rapidly during a “break-in” period of unknown length. Others state that marks on cartridge cases do not change, but that marks on bullets, especially lead bullets, do.So, right off the bat you can see that the "uniqueness" isn't so unique. Fingerprints, short of physical damage to the pad of a person's finger, don't change, but the markings produced on a cartridge case can - and from simple wear. Further, those changes may occur rapidly during the early use of the firearm, yet the baseline cartridge cases provided to the ballistics lab are one of if not the very first rounds fired from the gun.
But there are other problems as well. The technical evaluation portion explains:
When a cartridge is fired in a firearm, force of ignition will cause the firearm to leave various identifying marks on the cartridge case. These marks can be class, sub-class or individual characteristics. Class characteristics are features that indicate a restricted group source and result from design factors. Sub-class characteristics are features made during the course of manufacturing that further restrict the group source. On a fired cartridge case, sub-class characteristics can be mistaken for individual characteristics. Individual characteristics are those marks that serve to uniquely identify the cartridge case to only one gun. These marks can be made on different parts of the cartridge case by various parts of the firearm.That's important to remember: the software that is going to search through the BF database is pretty much limited to firing pin impressions, breech face marks, and ejector marks. BUT, there are problems with even these marks:
--
The firearm examiner can use any of these marks for identification; however, in most cases the areas used for identification are the following: breech face marks, firing pin impressions, extractor or chamber marks. For automated imaging, the only areas used for analysis are the firing pin impressions, breech face marks, and ejector marks. These are the marks that are typically repeatable and amenable to routine imaging. In most cases the firing pin may not leave sufficient detail for analysis and most examiners rely on the breech face marks.
The detail of these breech face impressions is dependent on cartridge chamber pressure and the type of breech face manufacture/condition. Lower pressure cartridges are not expected to consistently produce decent breech face impressions. Dirt or lead build up on the breech face can reduce the detail of breech face impressions.No wear and tear involved, this is dependent strictly on the pressure generated by the ammo being fired and/or by how dirty the gun is. The cartridge cases provided to the ballistics lab are fired from a brand-new, pristinely clean gun. But what if the gun is dirty? And you can bet the ammunition used by a perp won't be the same ammo used by the factory to provide the baseline cases.
Another variable in the production of breech face marks is the type of ammunition used. The detail left on a cartridge case is also dependent on the cartridge chamber pressure, bullet weight and the hardness of the primer. On some occasions, these can vary to such an extent that an examiner will not be able to identify test 1 to test 2 when different ammunition is used in the same gun. One of the cardinal rules in firearm examination is to test fire the gun with similiar ammunition as the evidence ammunition if at all possible.But when you're comparing to the baseline cartridge cases provided by the gun manufacturer, you don't have that option. If a trained forensic ballistics technician can't ID a case, what chance does a piece of software have? To give some idea of the variability, the report included this image of breechface marks on different manufacturer's ammunition fired from the same gun:
Here's the thing the gun
The difference, once again, is that the gun is already in police hands, and that gun provides multiple cases from similar ammunition to that found at the crime scene. With Maryland's and New York's IBIS systems, what they're trying to do is exactly backwards. All the ballistics technician has is one or more crime scene cases, and all he has to compare those cases to is an electronic database of two cases from each new handgun sold. The chance that the crime scene ammunition is of the same manufacture as the baseline is slim, and the same loading? Infinitesimal. For example, in 9mm alone, one of the most "popular" calibers used by criminals, Federal offers eleven different loads. Winchester offers twelve. Remington offers fourteen. Then there's CCI/Speer, Fiocci, Wolf, Black Hills, and many others, not to mention military surplus, commercial reloads, and even handloads. Even if you have ammunition from the same manufacturer, different loads will leave different marks, as the report noted: "The detail left on a cartridge case is also dependent on the cartridge chamber pressure, bullet weight and the hardness of the primer." Here's an example of two 7.65 (.32 Auto) Sellier & Bellot cases from the same gun using two different lots of the same load:
In addition, the cases from the suspect gun are being checked against a relatively small database of known crime scene evidence, not against a vast database of cases from every new handgun sold into circulation:
(S)ome firearms will reproduce (marks) well with sufficient detail, while some firearms will vary in their reproducibility depending on the cartridges used. Firing pins can be relatively smooth and nondescript. In such cases, these smooth firing pin marks can serve primarily as a class characteristic indicator. Even when they have gross features such as in the Bryco, these could be class or sub-class characteristics. An exception is the Glock firearm with its characteristic firing pin drag and aperture marks. This is one of the reasons that cartridge case examinations frequently involve the examination and comparison of other unique marks such as chamber marks, ejection port marks, ejector marks, and rimfire anvil marks. The impressions are not only dependent on the hardness of the primer, but also on how well the primer seats in the local cartridge case. In smaller databases these issues may not be significant, but with a large database using newly manufactured firearms, these differences can prove significant.(Underlining in the original, other emphasis mine.) I won't go into the technical specifics of this problem. If you want to know, read the report, Appendix A, but the concept shouldn't be hard to grasp. Instead of searching for a nail in a haystack, you're looking for a needle in a hay field. And with a million new handguns entering the market each year, that field just keeps getting bigger and bigger. Not to mention the 60+ million handguns already in circulation that aren't in the database. Not as simple as the Brady Bunch makes it sound, is it?
But wait! There's MORE!
The initial feasibility report spawned a later independent study report that expanded on the issues brought up. You see, the BATF was quite hurt by some of the statements made and took exception to the naysayers in California's Department of Justice, so a follow-on study was done to answer the items the BATF objected to. That report was written by Dr. Jan De Kinder, Ballistics Section Head of the National Institute for Forensic Science, in the Department of Justice for Belgium. (I guess they really wanted an unbiased observer.) Dr. De Kinder was a little less enthusiastic about how "unique" those markings were, as he stated in his introduction:
A ballistic imaging database system is based on the following premise: When a gun is fired, it may leave distinguishing marks on the fired bullet and cartridge case. A searchable database with images of such marks from all guns sold could be a valuable investigative tool. Such a database would permit linking of evidence bullets or cartridge cases back to the gun that fired them. Evidence found at a shooting incident would be scanned and compared to all entries in the database. Ideally, the system would provide investigators with the serial number of the firearm. The serial number can lead the investigators to the registered owner. The database could be called a “ballistic fingerprinting” database. If created, California’s ballistic fingerprinting database would quickly grow to be very large. More than 100,000 fired cartridges from new pistols would be added to it annually.Not as definitive as the Brady Center at all. Dr. De Kinder also explains how successful the initial tests were:
The AB1717 Evaluation was designed to test the performance of the IBIS™ system for the anticipated large database of new firearms. The experiment used 792 Smith & Wesson model 4006 semi-automatic pistols for this purpose. Each pistol was test fired using at least two cartridges of Federal brand ammunition and other ammunition. One of the test fired Federal cartridge cases for each of the pistols was registered into the database.The supporters of a BF database insist that the "unique fingerprint" will work to allow an automated system to identify the serial number of a gun used in a crime, but in controlled laboratory experiments it does not. And bear in mind, the ammunition tested was all from the same lot of Federal .40S&W caliber cartridges. The sample database had only 792 firearms all of the same make and model. This is the most ideal condition. Read the entire report if you're interested. (And if you've slogged this far down this dissertation, you must be interested!)
The duplicate Federal cartridge cases from fifty of these pistols were selected at random and compared with the database. The system ranks how well each entered mark matches the evidence. The higher the ranking the more similar the stored image is to the evidence’s mark. For the system to be successful, the correct gun should be listed in the top few ranks. The results show that 38 % of the fifty pistols were not listed in the top 15 ranks. The same experiments was repeated with ammunition of a different brands. In this case 62.5 % of the pistols were missed and not listed in the top 15 ranks. These results will be discussed in light of the investment in terms of equipment and personnel needed to set up a ballistic fingerprinting database. In fact, the trends in the obtained results show that the situation worsens as the number of firearms in the database is increased.
Now, I've discussed the problems involved in the BF database concerning only the ammunition used and normal wear, but what about alterations to the firearm, whether intentional or innocent? Dr. De Kinder stated:
Whereas the BATF sees altering a firearm as a non-issue, it is a real problem: Any reduction in the potential of 'hits' such as caused by alteration to a firearm is of concern when evaluating the usefulness from a technical point of view of a 'gun sales database'.It certainly would be if a BF database were known to be in use. Remember, the marks an automated system compares are the breechface, firing pin, and ejector marks. Does this bring anything to your attention? It should. Those marks are pretty much limited to semi-automatic handguns. Revolvers don't leave ejector marks. In fact, unless a perp reloads his revolver and drops the empty brass, a revolver doesn't leave cases at a crime scene at all - but revolvers are included in the database. So if you limit yourself to semi-auto handguns, how hard is it to change those marks?
Not hard at all. Take a Dremel tool and polish the breechface. Install a new firing pin. Grind just a little bit on the ejector with a stone or a file. If you want to get really creative, buy a new slide from Brownell's or from Numrich. After all, the part of the gun that carries the serial number is the frame. The slide is just another part. But it's the part that leaves the marks. If you're a professional criminal, why not? Or just use an older gun that won't be in the database. There's millions of them.
And that brings up another question: How old are guns used in crime? The gun
A) The gun used was a rifle and not a handgun. (TWO million new rifles enter circulation annually, as opposed to only ONE million handguns.)
B) The evidence collected at the crime scenes wasn't cartridge cases but projectiles, and the system doesn't track projectiles. Muhammed and Malvo did their shooting from inside a vehicle. The cartridge cases went with them when they escaped the crime scene.
C) The condition of the projectiles recovered (mostly jacket fragments and bullet bases) was enough to verify that the same gun was used, and that's about it. I'm pretty sure they did it more through metallurgical analysis than rifling marks. Certainly not enough of the bullets survived to have positively ID'd a firearm out of a huge database of firearms.
And, finally,
D) The gun was stolen, so even if they could have ID'd the weapon, all it would have told them was what gun shop it came from.
(Not exactly the whiz-bang tool it's played up to be, is it?)
Dr. De Kinder discussed the firearm age question, known in the business as "time to crime":
The time to crime (TTC) is an important parameter that can be used to reduce the size of the ballistic fingerprinting database. If this parameter drops down in a relatively short time period, the firearms which were sold long before the average TTC have a negligible chance of being used in crime. In other words, it allows one to minimize the retention time of the data in the ballistic fingerprinting database without losing a substantial amount of its performance.It would appear, then, that guns fresh off the showroom floor don't end up being used in a crime all that often.
In the Crime Gun Trace Reports 2000 from the ATF, average TTC are mentioned per age of the offender and type of firearm. The following results are obtained for semiautomatic pistols (4.5 years), revolvers (12.3 years), rifles (7.0 years), shotguns (7.6 years) and other firearms (7.1 years). The nationwide average TTC for all firearms for all ages of offenders is 6.1 years. As this study averages over the whole U.S.A., regional differences can be expected. The study particularly mentions Stockton, CA (9.2 years); San Jose, CA (9.0 years); Anaheim, Long Beach and Santa Ana (8.0 years) and Oakland, CA (8.0 years) as cities where the median time-to-crime is much longer than the overall city average. As all these cities lie within California, one can expect the average TTC for the State of California to be higher than the national value. More detailed data is required to determine correctly the median TTC for pistols in California.
So, let's review.
I: The system is limited to new handguns.If you didn't know any of this initially, then I can understand why Say Uncle's commenter wondered why the idea didn't work. It seems so simple. It's a fingerprint. It's "Gun DNA."
II: The system is limited to two cartridge cases per handgun.
III: Those cases are fired from the gun when it is brand-new and immaculately clean.
IV: Pretty much only semi-automatic handguns are going to leave cartridge cases at crime scenes, so the inclusion of revolvers seems a moot point.
V: Wear and tear from normal use can affect the markings left on the cartridge case, and such wear may be significant during the "break-in" period.
VI: Very few guns used in crime are anywhere near new.
VII: How dirty the firearm is can also affect the markings left on the case.
VIII: The ammunition used will dramatically affect the markings left on the case.
VIX: The automated system is pretty much limited to comparing the breechface, firing pin, and ejector marks.
IX: These marks can be altered quite simply, by wear and tear, by normal maintenance, by dirt and crud, or with specific intent.
X: As the size of the database goes up, the ability of the automated system to get a "hit" goes down.
XI: The size of the database will grow dramatically each and every year.
XII: In initial testing using a tiny database of identical firearms and ammunition from one lot, the success rate of the automated comparison system was less than 66%. When different ammunition was used, the success rate fell to well below 50%.
But it's not. Just like the gun SAFETY groups aren't what they say either. And here's the final nail in the coffin, from the last report that inspired all that commentary:
The MD-IBIS Program has collected and imported 43,729 cartridge cases into the system as of 9/7/04. It has incurred 208 criminal investigations leading to six (6) "hits" or matches. It should be reiterated at this point that the IBIS system does not make the match. It provides a means of narrowing the field of search. The examination by a competent Firearms Examiner on a known versus questioned basis provides the actual conclusion of a match. The estimated cumulative cost of the MD-IBIS Program to date is $2,567,633. The cost per "hit" value is $427,939, or approximately $60 per gun sold. None of the "hits" has been used in a criminal trial, and five (5) of the six (6) did not work according to the manner in which the system was designed. It actually functioned in reverse. The gun was already present. Although these hits may be interpreted by some as being in line with the Mission of the DSP, the mission accomplishment and cost effectiveness of the Program has not been demonstrated. The MD-IBIS Program has not lead to the solution or expediency of an investigation that could not have been accomplished by other traditional sources."It has simply failed in the Mission and Vision concepts originally established for the Program."
--
(T)o further review and evaluate MD-IBIS, a series of blind proficiency tests were given. The tests were designed to test the ultimate purpose of the Program. The double-blind tests (unknown to examiners and supervisors) began in July 2003. The basic design of the tests were to emulate actual situations involving the finding of spent cartridge cases and submit them to the Forensic Sciences Division/Firearms Toolmarks Unit for examination. Guns known to be included in the MD-IBIS Database were test fired and the cartridge cases collected. These cartridge cases were submitted as evidence in bogus criminal cases. There have been four blind proficiency tests submitted to date. There were no "hits" associated with any of the submitted proficiency tests.
--
The bottom line of this report is that the MD-IBIS System has failed to demonstrate the bottom line of the 1st report. The MD-IBIS Program, for all its good intentions, has not proven to be a time-saving tool for the Firearms Examiner or an investigative enhancement to the criminal investigator. It has simply failed in the Mission and Vision concepts originally established for the Program. Fiscal resources for the MD-IBIS program would be well spent in other Forensic Sciences Division programs...
So in addition to being useless, it has the extra added bonus of being expensive. The MD-IBIS report also notes that New York's system has been equally successful (i.e., a complete failure) but it's cost New York taxpayers over eight million dollars so far. According to this link, the price is nearly $16 million. It would appear that throwing more money at the problem doesn't help.
So the reaction I expect? The philosophy cannot be wrong. Try it again, ONLY HARDER!!All four ballistic database reports are available here:
Feasibility of a California Ballistics Identification System
AB1717 report - Technical Evaluation: Feasibility of a Ballistics Imaging Database for All New Handgun Sales
Maryland State Police Forensics Division IBIS report
MD-IBIS Progress Report #2
The OC Shooter's Page on Ballistic Fingerprinting - very comprehensive.
See also my earlier piece, Spin, Spin, Spin.
(Like this wasn't long enough already...)
UPDATE, 1/19: This is probably the most linked and most viewed piece I've written to date. Thanks to everyone who has commented and pointed readers this way (though I'm surprised I haven't seen a single hate-bomb out of the two thousand or so who've read this.) One commenter over at Chicago Boyz made an observation that I would have included in the body of this piece if it weren't already so long:
I've long since concluded that the gun controlers don't really expect measures like these to "work", in the sense of lowering crime, or aiding the police. The inconvenience and cost to gun owners is what they're aiming for. In the long run, if they can make gun ownership enough of a hassle, the number of gun owners in the next generation will decline, and perhaps we'll lose enough political clout that they can get the outright ban they really want.To which I replied:
But that's not a motive they can openly admit, and still hope to get the programs enacted.
Brett Bellmore hit the nail on the head - the purpose is not to build a useful tool for crimefighting, the purpose is to make it more difficult for individuals to aquire and keep firearms. The effort is, and the gun control groups state this implicitly, to reduce the number of guns, because they ALL blame "the number of guns" in circulation for the level of gun violence we have here.UPDATE: As of 2015 Maryland scrapped their IBIS system.
The British did a yeoman's job of making gun ownership an onerous privilege to exercise, then used the rules under which that privilege is exercised to remove whole classes of firearms...
From the law-abiding subjects. The criminals, however, still have access to pretty much anything they want.
Note this passage from the second Maryland report:
The Vision of the MD-IBIS Program was to have it fully performing in three to five years. This was to include the amassing of some 30,000 database images per year. This projection fell and was embarrassingly overstating. The actual acquisition of cartridge case numbers was about 36% of the estimation. The main reason for this drop-off appears to be from the reduction of handgun sales in Maryland after the passage of the law.Now, did those sales actually disappear, or did they go black-market? Did this law really reduce the influx of handguns into Maryland by nearly two-thirds, or did it make a whole bunch of Marylanders (choose to become) instant criminals?
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.