Repository navigation
TestGauge::test_time_block_decorator_with_label failure #942
Description
Activity
What is alpha?
What is alpha?
It's a DEC Alpha machine! I've seen issues show up here before due to (1) low clock resolution and (2) race conditions exposed by its slow speed. If the log isn't enough to identify the problem, I have free shell access to all of the unusual machines I test on. Would provisioning an account so you can investigate directly help?
One additional failure in 0.18.0.
================================================================== FAILURES =================================================================== _______________________________________________ TestGauge.test_time_block_decorator_with_label ________________________________________________ self = <tests.test_core.TestGauge testMethod=test_time_block_decorator_with_label> def test_time_block_decorator_with_label(self): value = self.registry.get_sample_value self.assertEqual(None, value('g2', {'label1': 'foo'})) with self.gauge_with_label.time() as metric: metric.labels('foo') > self.assertLess(0, value('g2', {'label1': 'foo'})) E AssertionError: 0 not less than 0.0 metric = <prometheus_client.context_managers.Timer object at 0x2000403df90> self = <tests.test_core.TestGauge testMethod=test_time_block_decorator_with_label> value = <bound method CollectorRegistry.get_sample_value of <prometheus_client.registry.CollectorRegistry object at 0x2000403db10>> tests/test_core.py:255: AssertionError ___________________________________________________ TestMultiProcess.test_gauge_mostrecent ____________________________________________________ self = <tests.test_multiprocess.TestMultiProcess testMethod=test_gauge_mostrecent> def test_gauge_mostrecent(self): g1 = Gauge('g', 'help', registry=None, multiprocess_mode='mostrecent') values.ValueClass = MultiProcessValue(lambda: 456) g2 = Gauge('g', 'help', registry=None, multiprocess_mode='mostrecent') g2.set(2) g1.set(1) > self.assertEqual(1, self.registry.get_sample_value('g')) E AssertionError: 1 != 2.0 g1 = prometheus_client.metrics.Gauge(g) g2 = prometheus_client.metrics.Gauge(g) self = <tests.test_multiprocess.TestMultiProcess testMethod=test_gauge_mostrecent> tests/test_multiprocess.py:194: AssertionErrorBoth failures are possibly related to timing things, which as you say might be due to a low clock resolution. I don't really know much about alpha, and don't have time to try to debug it. I would be happy to accept a contribution to fix the test failures though!
Would a focused test-only change for test_time_block_decorator_with_label still be useful? It currently expects two real clock readings around a very short block to produce a positive duration. I could use controlled clock values and assert the duration recorded on the labeled gauge, so the test doesn’t depend on the machine’s clock resolution. I haven’t reproduced this on Alpha, and would leave the separate multiprocess failure out of this first change. Is anyone already working on it?
@fahadaldajani92’s approach for test_time_block_decorator_with_label makes sense, so I’ll leave that one to him. I’d like to take the other failure from this thread, test_gauge_mostrecent (and test_gauge_livemostrecent, which has the same issue). The two set() calls can get identical time.time() stamps on a coarse clock, and on a tie the collector keeps whichever .db file it reads first, so the result depends on filesystem order. I’d patch time.time in the tests with increasing values. Test-only change; I’ll open a separate PR unless someone objects.
Hi, I am running the test suite on some obscure architectures and observed a failure on alpha, with one test:
This is fully reproducible, and I tested on both python3.10 and python3.11 with same results.
Complete log: 41y0768.log
Any help on why this test could be failing? I know this is an unusual platform, so I can provide shell access to the machine in question if somebody would mind taking a look. I can also interactively debug, or test patches as needed.
Thank you!