{"id":1996,"date":"2026-06-20T04:19:03","date_gmt":"2026-06-20T03:19:03","guid":{"rendered":"http:\/\/htor.inf.ethz.ch\/blog\/?p=1996"},"modified":"2026-06-28T08:29:22","modified_gmt":"2026-06-28T07:29:22","slug":"stop-saying-flops-a-tiny-rant-about-a-very-real-unit-problem","status":"publish","type":"post","link":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/2026\/06\/20\/stop-saying-flops-a-tiny-rant-about-a-very-real-unit-problem\/","title":{"rendered":"Stop Saying FLOPS: A Tiny Rant About a Very Real Unit Problem"},"content":{"rendered":"<article class=\"blog-post\">\n<header>\n<font size=\"+2\"><em>A modest proposal for saving scientific computing from typographic entropy.<\/em><\/font><\/p>\n<p><i>(This post is a summary of the discussion of rigorous performance notation and benchmarking methodology in our paper <a href=\"https:\/\/htor.inf.ethz.ch\/publications\/index.php?pub=222\">Scientific Benchmarking of Parallel Computing Systems<\/a> (Section 2.1.2) and the common criminal misuse of terms in our communities.)<\/i><\/p>\n<\/header>\n<section>\n<p style=\"line-height: 1.65; margin-top: 0; margin-bottom: 1em;\">\nHigh-performance computing has given humanity many beautiful things: weather forecasts, protein simulations, large-scale machine learning, and the ability to turn an electricity bill into a leaderboard entry. It has also given us one small but surprisingly persistent unit-writing mess: <strong>FLOP<\/strong>, <strong>FLOPS<\/strong>, <strong>flop<\/strong>, <strong>flops<\/strong>, and the mysterious creature known as <strong>flop\/s<\/strong>.<\/p>\n<p>The problem is not that people do not know what they mean. Usually they do. The problem is that they often write it in a way that mixes up a <em>count<\/em> of floating-point operations with a <em>rate<\/em> of floating-point operations per second. That is rather like mixing up joules and watts, except with more GPUs and fewer household appliances.<\/p>\n<p>So let us ask a deliberately pedantic question: <strong>what if \u201cfloating point operation\u201d were treated as a proper unit?<\/strong> If it were, the clean notation would be simple:<\/p>\n<ul>\n<li><strong>flop<\/strong> for a count of floating-point operations.<\/li>\n<li><strong>flop\/s<\/strong> for floating-point operations per second.<\/li>\n<li>Not <strong>FLOP<\/strong>, not <strong>FLOPS<\/strong>, and not the informal plural <strong>flops<\/strong>.<\/li>\n<\/ul>\n<p>This is not because lowercase letters are morally superior, although some metrologists may quietly suspect so. It is because scientific notation works best when symbols are stable, unambiguous, and boring in exactly the right way.<\/p>\n<\/section>\n<section>\n<h2>The unit-symbol rule: symbols are not abbreviations<\/h2>\n<p>The International System of Units, maintained by the BIPM, exists to make measurement communication consistent across science, technology, industry, and trade. The BIPM publishes the <a href=\"https:\/\/www.bipm.org\/en\/publications\/si-brochure\/\">SI Brochure<\/a> as the official reference for the system. NIST summarizes the relevant writing rules very clearly: unit symbols are written in lowercase unless they are derived from a person\u2019s name, such as <code>W<\/code> for watt or <code>Pa<\/code> for pascal.<\/p>\n<p>That rule alone already makes <strong>FLOP<\/strong> suspicious. If <em>flop<\/em> were a unit symbol, what famous scientist named \u201cFlop\u201d are we honoring? Florence L. O. Processor? Friedrich Ludwig Operand von Pipeline? No. There is no Person of Flop. Therefore, by analogy with ordinary SI-style unit symbols, the symbol should be lowercase: <strong><code>flop<\/code><\/strong>.<\/p>\n<p>NIST also states that unit symbols are never pluralized: <code>250 mm<\/code> means 250 millimeters, not <code>250 mms<\/code>. This is the key point. If <code>flop<\/code> is treated as a unit symbol, then <code>10 flop<\/code>, <code>10<sup>15<\/sup> flop<\/code>, and <code>1.2e21 flop<\/code> are correct in the same way that <code>10 m<\/code>, <code>10 kg<\/code>, and <code>10 J<\/code> are correct. The quantity changes; the symbol does not.<\/p>\n<p>The distinction is not just stylistic. NIST\u2019s SI guidance emphasizes that unit symbols are symbols, not ordinary English abbreviations. They obey mathematical-symbol rules, not plural-noun rules. In other words, a unit symbol is not a little word that needs an <code>s<\/code> when it becomes lonely in a crowd. It is a symbol. It remains calm.<\/p>\n<\/section>\n<section>\n<h2>\u201cWatts\u201d, \u201ckgs\u201d, and other small crimes against clarity<\/h2>\n<p>Consider a few everyday examples. In formal technical writing, we would normally write:<\/p>\n<ul>\n<li><code>2000 W<\/code>, not <code>2000 Watts<\/code> when using the symbol.<\/li>\n<li><code>2 kg<\/code>, not <code>2 kgs<\/code>.<\/li>\n<li><code>10 km<\/code>, not <code>10 kms<\/code>.<\/li>\n<\/ul>\n<p>There is a small nuance here: spelling out a unit name as an English noun can require a plural, so \u201c2000 watts\u201d is grammatically fine in prose. NIST explicitly says plural unit names are used when English grammar requires them. But unit <em>symbols<\/em> do not get plural endings: <code>W<\/code>, <code>kg<\/code>, and <code>km<\/code> remain unchanged regardless of the number.<\/p>\n<p>This is why <strong>\u201cmany flop\u201d<\/strong> is the right style if <code>flop<\/code> is the unit symbol. It may sound odd at first, just as \u201c12 fish\u201d sounds odd if you were expecting \u201cfishes,\u201d but metrology is not here to optimize for vibes. It is here to prevent ambiguity, preferably before the benchmark marketing department discovers it.<\/p>\n<\/section>\n<section>\n<h2>The real confusion: flop versus flop\/s<\/h2>\n<p>The strongest reason to avoid <strong>flops<\/strong> is that it collides with the traditional acronym <strong>FLOPS<\/strong>, commonly expanded as \u201cfloating-point operations per second.\u201d Industry articles still explain FLOPS as a rate, not the plural of a single operation. That means \u201cflops\u201d is a dangerously overloaded spelling: it can be read as a plural count of operations or as a rate of operations per second. Those are not the same quantity.<\/p>\n<p>A count and a rate differ by a dimension of time. <code>10<sup>18<\/sup> flop<\/code> is a number of operations. <code>10<sup>18<\/sup> flop\/s<\/code> is a speed. Confusing them is like confusing \u201cI drove 100 km\u201d with \u201cI drove 100 km\/h.\u201d One tells you how far you went. The other tells you how fast you were going. Only one of them will impress the traffic police.<\/p>\n<p>The ambiguity is not theoretical. AMD\u2019s ROCm documentation notes that terms such as peak FLOPs, max-achievable FLOPs, and delivered FLOPs have historically been used interchangeably, creating confusion and incorrect comparisons. The same source defines peak FLOPs through a formula involving frequency, cores, and operations per cycle, i.e., a rate-like hardware capability rather than a mere count of operations.<\/p>\n<p>This is exactly why <strong><code>flop<\/code><\/strong> and <strong><code>flop\/s<\/code><\/strong> are preferable. They make the dimensional distinction visible:<\/p>\n<ul>\n<li><code>3.4e12 flop<\/code>: a workload size, operation count, or algorithmic cost.<\/li>\n<li><code>3.4e12 flop\/s<\/code>: a performance rate.<\/li>\n<\/ul>\n<p>The slash is doing real work. It is not decoration. It is the tiny typographic hinge on which dimensional sanity swings.<\/p>\n<\/section>\n<section>\n<h2>But isn\u2019t \u201cFLOP\u201d an acronym?<\/h2>\n<p>Historically, yes: people often write <strong>FLOP<\/strong> for \u201cfloating-point operation\u201d and <strong>FLOPS<\/strong> for \u201cfloating-point operations per second.\u201d That convention is widespread. It is also exactly the convention that creates the mess.<\/p>\n<p>If we are writing ordinary English prose, acronyms are fine. But if we are writing values of quantities, it is better to behave like we are using units. NIST\u2019s SI guidance recommends expressing quantity values using Arabic numerals paired with unit symbols. Once we decide that a floating-point operation count is a measurable quantity, the natural notation is not acronym style but unit-symbol style.<\/p>\n<p>That means:<\/p>\n<ul>\n<li><strong>Use <code>flop<\/code><\/strong> when counting floating-point operations.<\/li>\n<li><strong>Use <code>flop\/s<\/code><\/strong> when reporting floating-point operations per second.<\/li>\n<li><strong>Avoid <code>FLOP<\/code><\/strong> because unit symbols are not capitalized unless derived from names.<\/li>\n<li><strong>Avoid <code>flops<\/code><\/strong> because unit symbols are not pluralized and because it is visually confusable with <code>flop\/s<\/code>.<\/li>\n<li><strong>Avoid <code>FLOPS<\/code><\/strong> when you can write the dimensionally explicit <code>flop\/s<\/code>.<\/li>\n<\/ul>\n<\/section>\n<section>\n<h2>Recommended style guide<\/h2>\n<table>\n<thead>\n<tr>\n<th>Meaning<\/th>\n<th>Recommended<\/th>\n<th>Avoid<\/th>\n<th>Why<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>One floating-point operation<\/td>\n<td><code>1 flop<\/code><\/td>\n<td><code>1 FLOP<\/code><\/td>\n<td>Lowercase follows unit-symbol style unless derived from a proper name.<\/td>\n<\/tr>\n<tr>\n<td>Many floating-point operations<\/td>\n<td><code>10<sup>9<\/sup> flop<\/code><\/td>\n<td><code>10<sup>9<\/sup> flops<\/code><\/td>\n<td>Unit symbols are not pluralized.<\/td>\n<\/tr>\n<tr>\n<td>Floating-point operation rate<\/td>\n<td><code>10<sup>9<\/sup> flop\/s<\/code><\/td>\n<td><code>10<sup>9<\/sup> FLOPS<\/code>, <code>10<sup>9<\/sup> flops<\/code><\/td>\n<td><code>flop\/s<\/code> explicitly shows \u201coperations per second\u201d and avoids count\/rate ambiguity.<\/td>\n<\/tr>\n<tr>\n<td>Metric-prefixed rate<\/td>\n<td><code>1 Tflop\/s<\/code><\/td>\n<td><code>1 TFLOPS<\/code>, <code>1 teraflops<\/code><\/td>\n<td>SI prefixes attach directly to symbols without spaces, and symbols are not pluralized.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/section>\n<section>\n<h2>A few examples<\/h2>\n<p>Instead of:<\/p>\n<blockquote><p>The model required <strong>3e23 FLOPs<\/strong> to train and achieved <strong>500 TFLOPS<\/strong>.<\/p><\/blockquote>\n<p>Write:<\/p>\n<blockquote><p>The model required <strong>3e23 flop<\/strong> to train and achieved <strong>500 Tflop\/s<\/strong>.<\/p><\/blockquote>\n<p>Instead of:<\/p>\n<blockquote><p>This kernel performs <strong>two FLOPS<\/strong> per cycle.<\/p><\/blockquote>\n<p>Write:<\/p>\n<blockquote><p>This kernel performs <strong>2 flop\/cycle<\/strong>.<\/p><\/blockquote>\n<p>Instead of:<\/p>\n<blockquote><p>We counted <strong>100 megaFLOPs<\/strong>.<\/p><\/blockquote>\n<p>Write:<\/p>\n<blockquote><p>We counted <strong>100 Mflop<\/strong>.<\/p><\/blockquote>\n<p>The point is not to win an argument at the typography conference, although that is a noble side quest. The point is to make the units carry the meaning precisely.<\/p>\n<\/section>\n<section>\n<h2>The punchline<\/h2>\n<p>If floating-point operations were treated as a real unit, the notation would be beautifully simple:<\/p>\n<p style=\"font-size: 1.2em; font-weight: bold;\">Use <code>flop<\/code> for work, and <code>flop\/s<\/code> for speed.<\/p>\n<p>Everything else is either shouting, pluralizing a symbol, or hiding a division by seconds in an acronym. <strong>FLOP<\/strong> looks like an acronym. <strong>FLOPS<\/strong> looks like a plural and a rate at the same time. <strong>flops<\/strong> looks casual, but it quietly smuggles ambiguity into the room. And ambiguity, like thermal throttling, usually arrives just when the benchmark is about to get interesting.<\/p>\n<p>So let us be precise. Let us be boring in the way good notation is boring. Let us write:<\/p>\n<ul>\n<li><code>1 flop<\/code><\/li>\n<li><code>10<sup>15<\/sup> flop<\/code><\/li>\n<li><code>10<sup>15<\/sup> flop\/s<\/code><\/li>\n<\/ul>\n<p>Your readers will understand the difference between a count and a rate. Your units will stop wearing unnecessary capital letters. And somewhere, perhaps, a metrologist will smile quietly into a perfectly lowercase cup of coffee.<\/p>\n<p>For readers interested in the broader question of how to measure, report, and compare parallel computing performance rigorously, the paper <a href=\"https:\/\/htor.inf.ethz.ch\/publications\/index.php?pub=222\">Scientific Benchmarking of Parallel Computing Systems<\/a> contains a much more detailed treatment of benchmarking methodology, performance analysis, and the many ways seemingly simple metrics can mislead.<\/p>\n<\/section>\n<footer>\n<h2>References<\/h2>\n<ol>\n<li>BIPM, <em>The International System of Units (SI), 9th edition<\/em>, official SI Brochure page, updated 2026: <a href=\"https:\/\/www.bipm.org\/en\/publications\/si-brochure\/\">https:\/\/www.bipm.org\/en\/publications\/si-brochure\/<\/a><\/li>\n<li>NIST, <em>Writing with SI (Metric System) Units<\/em>, including rules that unit symbols are lowercase unless derived from a person\u2019s name, are not pluralized, and are written with a space after the number: <a href=\"https:\/\/www.nist.gov\/pml\/owm\/writing-si-metric-system-units\">https:\/\/www.nist.gov\/pml\/owm\/writing-si-metric-system-units<\/a><\/li>\n<li>NIST, <em>Guide to the SI, Chapter 9: Rules and Style Conventions for Spelling Unit Names<\/em>, including rules for capitalization and plural unit names: <a href=\"https:\/\/www.nist.gov\/pml\/special-publication-811\/nist-guide-si-chapter-9-rules-and-style-conventions-spelling-unit-names\">https:\/\/www.nist.gov\/pml\/special-publication-811\/nist-guide-si-chapter-9-rules-and-style-conventions-spelling-unit-names<\/a><\/li>\n<li>AMD ROCm Blogs, <em>Understanding Peak, Max-Achievable &amp; Delivered FLOPs<\/em>, noting historical confusion among peak, max-achievable, and delivered FLOPs terminology: <a href=\"https:\/\/rocm.blogs.amd.com\/software-tools-optimization\/Understanding_Peak_and_Max-Achievable_FLOPS\/README.html\">https:\/\/rocm.blogs.amd.com\/software-tools-optimization\/Understanding_Peak_and_Max-Achievable_FLOPS\/README.html<\/a><\/li>\n<li>BizTech Magazine, <em>What Is FLOPS and How Is It Used in Supercomputing?<\/em>, explaining that FLOPS means floating-point operations per second and is not simply the plural of FLOP: <a href=\"https:\/\/biztechmagazine.com\/article\/2023\/08\/what-flops-and-how-does-it-help-supercomputer-performance-perfcon\">https:\/\/biztechmagazine.com\/article\/2023\/08\/what-flops-and-how-does-it-help-supercomputer-performance-perfcon<\/a><\/li>\n<\/ol>\n<\/footer>\n<\/article>\n","protected":false},"excerpt":{"rendered":"<p>A modest proposal for saving scientific computing from typographic entropy. (This post is a summary of the discussion of rigorous performance notation and benchmarking methodology in our paper Scientific Benchmarking of Parallel Computing Systems (Section 2.1.2) and the common criminal &hellip; <a class=\"more-link\" href=\"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/2026\/06\/20\/stop-saying-flops-a-tiny-rant-about-a-very-real-unit-problem\/\">read more<\/a><\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1996"}],"collection":[{"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=1996"}],"version-history":[{"count":28,"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1996\/revisions"}],"predecessor-version":[{"id":2031,"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1996\/revisions\/2031"}],"wp:attachment":[{"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=1996"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=1996"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/htor.inf.ethz.ch\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=1996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}