This table presents empirical findings of RQ3. Among the 577 defective APIs, we find clear clues to the root causes of inducing pipelines’ performance inconsistencies, crashes and NaN bugs for 270 APIs (46.8%), in tde corresponding library release notes, issues and code commits. We identified four types of essential root causes and discuss them in two scenarios: performance impacts induced by one library version and performance impacts induced by a combination of library versions.
| Root Causes | Crash | Execution Time | Competition Score | Memory Usage | NaN |
|---|
| #Variants | #Pipelines | #Variants | #Pipelines | #Variants | #Pipelines | #Variants | #Pipelines | #Variants | #Pipelines |
|---|
| Performance Impacts induced by one library version | API Optimization | Hardware support | - | - | 92 | 5 | 7 | 2 | 12 | 1 | - | - |
|---|
| Data Processing | - | - | 501 | 11 | 470 | 14 | 537 | 13 | - | - |
| Calculations | - | - | 675 | 18 | 429 | 11 | 254 | 15 | - | - |
| Otders | - | - | 41 | 2 | 8 | 1 | 15 | 3 | - | - |
| Sum | - | - | 1,309 | 36 | 916 | 28 | 818 | 32 | - | - |
| Default Hypermeter Change | - | - | 30 | 5 | 25 | 2 | 17 | 7 | 9 | 2 |
| Technical Debt | - | - | 47 | 2 | 9 | 1 | 92 | 2 | - | - |
| API Breaking Change | 619 | 35 | - | - | - | - | - | - | 236 | 12 |
| Performance Impacts induced by a combination of library versions | Indirect Dependency Interference | 31 | 7 | 991 | 15 | 22 | 5 | 421 | 11 | 6 | 3 |
|---|
| Data Flow Interference | 17 | 3 | 1,114 | 24 | 661 | 19 | 501 | 16 | 4 | 3 |
Help ×
We provide a list of PLC issues captured by us in real-world pipelines and popular ML libraries.
- Go to Empirical Findings page
- Select a bug and click on reproduce result link.
- You can find the reproduction results of each version and the related reproduction code.
![avatar]()
