Odd things that astound scholastics attempting to market a static checking apparatus. .


39 views
Uploaded on:
Category: Business / Finance
Description
Weird things that surprise academics trying to commercialize a static checking tool. Andy Chou, Ben Chelf, Seth Hallem Charles Henri-Gros, Bryan Fulton, Ted Unangst Chris Zak Coverity Dawson Engler Stanford. A naïve view. Initial market analysis:
Transcripts
Slide 1

Odd things that astonish scholastics attempting to market a static checking instrument. Andy Chou, Ben Chelf, Seth Hallem Charles Henri-Gros, Bryan Fulton, Ted Unangst Chris Zak Coverity Dawson Engler Stanford

Slide 2

A guileless view Initial market examination: "We handle linux, bsd, we simply require a pretty box!" Not exactly. In the first place run of static examination: no check, no bug. Two first request cases we never would have speculated. Issue 1: on the off chance that you can\'t discover the code, can\'t check it. Issue 2: in the event that you can\'t gather code, you can\'t check it. And afterward: how to profit on programming instrument? "Apparatuses. Huh. Apparatuses are hard." Any VC in mid 2000.

Slide 3

Myth: the C (or C++) dialect exists. All things considered, not by any stretch of the imagination. The standard is not a compiler. What exists: gcc-2.1.9-ac7-prepatch-alpha, xcc-i-didn\'t comprehend pages4,33,208-242-of-standard. Goodness. What\'s more, Microsoft. Conformance = aggressive impediment. Crunch the numbers on how this misshapes .c documents Basic LALR law: What can be parsed will be composed. Run: static examination must arrange code to check. In the event that you can\'t (accurately) parse "dialect" can\'t check. Regular (mis)usage show: "supposedly C" header record accomplishes something odd not-C thing. Included by all source. Client watches your compiler emanate voluminous parse blunders. (This is not great.) obviously: deteriorates with C++ (which we bolster)

Slide 4

Some awful cases to discover in headers Banal. In any case, take additional time than you can trust: And, obviously, asm: short x; int *y = &(int)x ; int foo( int an, int a ); void x; unsigned x @ "Content";/unless "- pressed"! __packed (… ) struct foo { … } unsigned x = 0xdead_beef; End lines with "\r" instead of "\n"/newline = end __asm mov eax, eab/"]" = end __asm [ mov eax, eab ] #pragma asm mov eax, eab #pragma end_asm asm foo() { mov eax, eab; }

Slide 5

Microsoft case: precompiled headers Spec: Implication It deteriorates: on-the-fly header creation The compiler treats all code happening before the .h record as precompiled. It skips to simply past the #include mandate connected with the .h record, utilizes the code contained as a part of the .pch document, and afterward accumulates all code after filename I can put whatever I need here. It doesn\'t need to assemble. On the off chance that your compiler gives a blunder it sucks. #include <some-precompiled-header.h>

Slide 6

Solution: pre-preprocessing revise rules. Supply consistent expressions to modify terrible builds #pragma asm … #pragma end_asm ppp_translate ("/#pragma asm/#if 0/"); ppp_translate("/#pragma end_asm/#endif/"); #if 0 … #endif

Slide 7

What this all methods solidly. We utilize Edison Design Group (EDG) frontend Pretty much everybody employments. Been around since 1989. Forceful support for gcc, microsoft, and so forth (bug compat!) Still: coverity by a wide margin the biggest wellspring of EDG bugs: 146 parsing experiments (i.e., we got scorched) 219 compiler line interpretation test cases (i.e., in the same place). 163 spots where frontend hacked ("#ifdef COVERITY") Still require custom rewriter for some upheld compilers: 205 hpux_compilers.c 215 iar_compiler.c 240 ti_compiler.c 251 green_hills_compiler.c 377 intel_compilers.c 453 diab_compilers.c 453 sun_compilers.c 485 arm_compilers.c 617 gnu_compilers.c 748 microsoft_compilers.c 1587 metrowerks_compilers.c …

Slide 8

Academics don\'t comprehend cash. "We\'ll simply charge per situate like other people" Finish the story: "Organization X purchases three Purify seats, one for Asian, one for Europe and one for the US… " Try #2: "we\'ll charge per lines of code" "That is a truly dumb thought: (1) … , (2) … , … (n) … " Actually works. I\'m still in stun. Would prescribe it. Great component for vender: No seat diversions. Income develops with code measure. Keep running on another code base = new deal. Great component for purchaser: No seat-demonstrate issues Buy once for venture, then done. No per-situate or per-use cost; no hub bolt issues; no issues including, evacuating or renaming engineers (or machines) People really appear to like this pitch.

Slide 9

Some experience. Astonish: Sales folks are extraordinary Easy to assess. Secluded. Organization X purchases apparatus, then cuts back. Great or awful? For deals, great: X fires 110 individuals. They land positions somewhere else. Prescribe coverity. 4 made it happen. Extensive organizations "need" to be straightforward Veritas: need checking so don\'t accidently disregard! What would you be able to offer? Client not same as device manufacturer. Guileless. Careless. Barbarous. Makes it hard to convey anything modern. Case: factual derivation. Some ways, checkers linger much behind our examination ones.

Slide 10

"No, your instrument is broken: that is not a bug" for(s=0; s < n; s++) { … switch(s) { case 0: assert(0); return; … } … dead code… "No, the circle will go through once!" "No, && is \'or\'!" "No, ANSI gives you a chance to compose 1 past end of the exhibit!" ("We\'ll need to settle on a truce." !!!!) for(i=1; i < 0; i++) { … deadcode… } void *foo(void *p, void *q) { if(!p && !q) return 0; unsigned p[4]; p[4] = 1;

Slide 11

Breakthrough innovation out of Stanford Company fused Product development and expansion Achieved gainfulness 2000 2002 2003 2004-05 Version 2.0 item discharged. Organization quadruples 70+ clients including Juniper, Synopsys, Oracle, Veritas, nVidia, palmOne. Self supported Meta-level gathering checker ("Stanford Checker") distinguishes 2000+ bugs in Linux. 7 early adopter clients, including VMWare, SUN, Handspring. Coverity accomplishes productivity. Downpour of solicitations from organizations needing access to the new innovation. To start with client signs: Sanera frameworks Coverity\'s business history

Slide 12

EDA Security Government An incomplete rundown of 70+ clients… Storage Networking Embedded Biz Applications OS Open Source

Slide 13

Summary Static investigation Better at checking surface properties Big wins: don\'t run code, all ways. Simple conclusion. Low incremental cost per line of code Can get brings about an evening: much simpler to popularize. 10-100x more bugs. Show checking Better at checking code suggestions. Real win over testing: investigate all activities a state can do before going to next Makes low-likelihood occasions as plausible as high. Works exceptionally well when enormous interleavings and bugs loathsome.

Slide 14

Open Q: how to get the bugs that matter? Myth: all bugs matter and all will be altered *FALSE* Find 10 bugs, all get settled. Find 10,000… Reality All destinations have numerous open bugs (saw by us & PREfix) Myth lives since condition of-workmanship is so awful at bug finding What clients truly need: The 5-10 that "truly matter" General conviction: bugs take after 90/10 appropriation Out of 1000, 100 record for generally torment. Altering 900 misuse of assets & may exacerbate the situation How to discover most exceedingly bad? Nobody has a clever response to this.

Slide 15

Bugs discovered Bugs that mattered Bugs that mattered The trust Bugs found The invalid theory Bugs that mattered Bugs found A Possibility Open Q: Do static apparatuses truly offer assistance? Risks: Opportunity cost. Deterministic bugs to non-deterministic.

Slide 16

Some superficial static investigation encounters Bugs are wherever Initially stressed we\'d depend on verifiable information… 100 checks? You\'ll discover bugs (if not, bug in investigation) Finding blunders frequently simple, saying why is hard Have to track and well-spoken all reasons. Simplicity of-review *crucial* Extreme: Don\'t report blunders that are too hard. The benefit of checking human-level operations Easy for individuals? Simple for investigation. Hard for examination? Hard for individuals. Soundness not required for good results.

Slide 17

Myth: more investigation is constantly better Does not generally enhance comes about, and can aggravate The best blunder: Easy to analyze True mistake More examination utilized, the more regrettable it is for both More examination = the harder mistake is to reason about, since client needs to physically copy every examination step. Number of steps increment, so does the shot that one turned out badly. No examination = no mix-up. Practically speaking: Demote mistakes in view of how much investigation required Revert to weaker examination to filter out simple bugs Give up on blunders that are too difficult to analyze.

Slide 18

Myth: Soundness is an ideals. Soundness: Find all bugs of sort X. Not an awful thing. More bugs great. Yet, can just do in the event that you check powerless properties. What soundness truly needs to be the point at which it grows up: Total rightness: Find all bugs. Most immediate estimation: find whatever number bugs as could be allowed. Opportunity cost: Diminishing returns: Initial examination finds most bugs Spend on what gets the following greatest arrangement of bugs Easy trial: bug means sound versus unsound devices. End-to-end contention: "It for the most part does not bode well to diminish the remaining mistake rate of one framework segment (property) much underneath that of the others.

Recommended
View more...