Phone Number Generator: Test Log Redaction Before Sharing Failures

The boundary is the shared log, not the fixture generator

A phone number generator can supply a fictional input while a test still exposes the same input in a failure report. Today we test that reporting boundary. We keep the generator's output unchanged and build a smaller public event from reviewed fields. Our broader masking and logging guide covers UI, storage and access boundaries; this tutorial measures an emitted-output contract. This does not change number allocation, CSV export, schema migration or download integrity.

Our baseline phone is +12025550123, the US example in our source-pinned dataset. The existing dataset marks its limited fictional 555-01xx range and links NANPA. INERT_OTP_CANARY_742913 is invented text, not an issued verification code. Neither value identifies a real contact or triggers a message. We use these canaries only to detect exposure in memory-captured output.

Write a small output policy before adding a regex

This is our narrow tutorial policy, not a general detector for personal information. A date-shaped version string is a syntactic constraint, not evidence that a release exists. Replay fields must come from reviewed test metadata, never from arbitrary request content. Review any policy expansion before allowing new output fields.

Our six measured output cases

We ran the original Python experiment on Python 3.13.15 on October 10, 2026. Each assertion examines the string written by StreamHandler, not only an intermediate dictionary. The complete measured report is downloadable.

All seven captured sink outputs excluded both exact canaries. Four accepted cases retained seed 417, path 3:1:0 and dataset version 2026-07-20. The argument and boolean-seed cases emitted static rejection events without replay metadata; a rejected input is not a successful replay report. The producer object remained unchanged.

Two negative controls establish why inspecting only record.msg is insufficient. A normal formatter emitted the phone supplied through %s, despite the message template containing no phone. A normal exception formatter emitted the inert OTP from the exception, despite the message containing no OTP. These are measured toy cases, not an estimate of production leak rates.

Build a new event at each reviewed output handler

Our formatter reads the structured message directly, projects it onto the small output schema and serializes that new event. It does not call record.getMessage() or the base formatter, render arbitrary objects, append cached exception text, or mutate the shared record. Unknown nested debug values are never recursively copied into the public event. The accepted baseline output is:

{
  "event": "fixture_failure",
  "error_code": "PARSER_REJECTED",
  "replay": {
    "seed": 417,
    "counterexample_path": "3:1:0",
    "dataset_version": "2026-07-20"
  },
  "fixtures": [
    {
      "iso2": "US",
      "e164Example": "[REDACTED]",
      "delivery_allowed": false
    }
  ]
}

Python's logging reference describes message/argument merging and additional exception information. It also explains handler routing and why a parent logger's filters do not automatically filter descendant events. Our output contract therefore sits on each handler we review. The tutorial creates one isolated logger, two in-memory handlers for one case, and propagate = False for that dedicated logger. It does not replace your application's logging configuration or suppress required operational events.

handler = logging.StreamHandler(captured_output)
handler.setFormatter(PublicFixtureFormatter())
dedicated_test_logger.addHandler(handler)
# Apply the reviewed formatter to EVERY sink in this logger's route.
# Inspect each emitted string, including error paths.

The script deliberately excludes raw tracebacks and source file paths from this shareable artifact. Keep any authorized internal diagnostic context under your existing access and retention rules. A log formatter cannot remove a secret already printed by another handler, a direct print, CI stdout, a tracing tool or an attachment collector.

Run the assertions before sharing a test failure

Download and inspect the standard-library script, then run:

python phone-fixture-log-redaction.py

It executes all six cases, checks both negative controls and prints only the result summary plus the redacted sample. It makes no network calls and does not read production logs. No dependency installation is needed. An assertion failure exits nonzero. A passing report covers these exact tutorial inputs and routes.

1. Inventory every publishing route. Include console/file handlers, ancestor handlers, CI artifacts and external monitoring. Our two handlers are an example, not a complete system inventory.
2. Use synthetic canaries. Exercise nested fixture lists, new aliases, formatting arguments, exception paths and malformed metadata.
3. Inspect emitted artifacts. Require one parseable JSON line, absent raw canaries, the intended event and exact approved replay values.
4. Keep reporting failures visible. An invalid input yields a static rejection event; do not dump the original input as a fallback.
5. Review the actual shareable bundle. Check attachments and independent output systems too. Stop sharing if the output contract fails.

OWASP's logging guidance calls for handling sensitive data appropriately and testing logging behavior. Its verification section includes failure cases and unintended effects. Our emitted-output checks implement a small test policy for this article; they do not certify production privacy, legal compliance or all possible secrets.

FAQ from an exception-redaction question

A public developer question describes sensitive URL information appearing in an exception log and asks where to transform the record. We use that one question as a signal for the exception case, not as a policy source, author-location claim or measure of frequency.

Is a fictional phone number enough to make a failure log shareable?

No. Inspect the entire artifact. It may contain other fields, exception strings, paths or independent attachments. Our example uses fictional inputs, but still tests a deliberately restricted output contract.

Why can cleaning the message template miss a number?

The final formatter may merge arguments into the message or append exception details. Our two negative controls reproduce those separate paths. Check the final emitted output.

Can I keep the seed and counterexample path?

Yes, if they come from your reviewed test metadata and satisfy the sharing policy. Our accepted cases preserve those fields exactly while masking the phone. Invalid replay metadata produces a rejection event.

What happens when a new field contains a phone or OTP?

Our projection omits every field not explicitly included in the public schema. The alias and nested cases verify this behavior. Adding a new retained field requires a policy and output-test review.

Does this formatter protect every log in my application?

No. It covers only handlers using it in the reviewed route. Inventory other handlers, propagation, direct output, tracing and artifact collection. There is no general personal-data detection guarantee.