{"id":1606,"date":"2026-04-11T15:01:56","date_gmt":"2026-04-11T15:01:56","guid":{"rendered":"https:\/\/abilit.eu\/?page_id=1606"},"modified":"2026-04-11T15:01:56","modified_gmt":"2026-04-11T15:01:56","slug":"cleanup-housekeeping-datapoint-cleaner","status":"publish","type":"page","link":"https:\/\/abilit.eu\/index.php\/offer\/concept-area\/cleanup-housekeeping-datapoint-cleaner\/","title":{"rendered":"Cleanup \u2014 Housekeeping &#038; Datapoint Cleaner"},"content":{"rendered":"<h2 class=\"wp-block-post-title\">Cleanup \u2014 Housekeeping &#038; Datapoint Cleaner<\/h2>\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-c7ebd8d6 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<h3 class=\"wp-block-heading\">Core capabilities<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Identify stale data points, orphaned sensors, and unused tag definitions based on configurable inactivity windows.<\/li>\n\n\n\n<li>Safe archival to compressed snapshot stores (with CRC\/SHA verification).<\/li>\n\n\n\n<li>Dry\u2011run and report mode with full change diffs and audit logs for compliance.<\/li>\n\n\n\n<li>Normalization and deduplication pipelines for tag metadata and historical series.<\/li>\n\n\n\n<li>Integration with backup and retention policies; supports tiered archival (S3, MinIO, on\u2011prem NAS).<\/li>\n<\/ul>\n<\/div>\n\n\n\n<div class=\"wp-block-column has-background is-layout-flow wp-block-column-is-layout-flow\" style=\"border-top-left-radius:42px;border-top-right-radius:42px;border-bottom-left-radius:42px;border-bottom-right-radius:42px;background-color:#f8fbff;padding-top:0;padding-bottom:0;flex-basis:33.33%\">\n<div class=\"wp-block-group has-global-padding is-layout-constrained wp-container-core-group-is-layout-094d544d wp-block-group-is-layout-constrained\" style=\"border-top-left-radius:27px;border-top-right-radius:27px;border-bottom-left-radius:27px;border-bottom-right-radius:27px;padding-top:var(--wp--preset--spacing--x-small);padding-right:var(--wp--preset--spacing--x-small);padding-bottom:var(--wp--preset--spacing--x-small);padding-left:var(--wp--preset--spacing--x-small)\">\n<h4 class=\"wp-block-heading\">Quick facts<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Package:<\/strong> Cleanup vA.B<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Modes:<\/strong> dry\u2011run, apply, schedule<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Download:<\/strong><a href=\"https:\/\/abilit.eu\/wp-admin\/&lt;!-- TODO: DOWNLOAD LINK --&gt;\">Get package<\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<h3 class=\"wp-block-heading\">How Cleanup works<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Cleanup analyses database records and time\u2011series meta information using configurable rules. It generates a change plan (human\u2011readable diff) and an auditable runbook. Administrators should always review the plan in dry\u2011run mode before applying changes. Archival moves data to a chosen cold store and replaces live records with compact pointers for recovery.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Scan phase \u2014 collect candidates based on inactivity thresholds, tag ownership and retention rules.<\/li>\n\n\n\n<li>Plan phase \u2014 generate human\u2011readable change plan with impact assessment and space gains.<\/li>\n\n\n\n<li>Review &amp; Approval \u2014 export plan as JSON\/HTML for audit and sign\u2011off.<\/li>\n\n\n\n<li>Apply phase \u2014 archive selected records, update indexes, and emit verification checksums.<\/li>\n\n\n\n<li>Post\u2011apply verification \u2014 run integrity checks and report results to monitoring and SIEM.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Configuration highlights<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Rule engine: regex and metadata conditions, owner whitelists, tag classification.<\/li>\n\n\n\n<li>Inactivity window: configurable per tag class (hours\/days\/months).<\/li>\n\n\n\n<li>Archival backends: S3, MinIO, NFS, CIFS \u2014 configurable lifecycle policies.<\/li>\n\n\n\n<li>Retention policies: soft\u2011delete grace windows and automatic pointer creation.<\/li>\n\n\n\n<li>Compliance: immutable archival support (WORM) for regulated environments.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Dry\u2011run &amp; Auditability<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Every Cleanup run produces:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Full change plan (JSON + HTML) summarizing records, sizes and expected space reclaimed.<\/li>\n\n\n\n<li>Checksums (SHA\u2011256) for archived bundles and a verification manifest.<\/li>\n\n\n\n<li>Audit log entries with operator, timestamp and approval token (supports OIDC SSO integration).<\/li>\n\n\n\n<li>Revert plan enabling safe restoration from archive within defined SLAs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Integrations &amp; Notifications<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Cleanup can emit events to monitoring stacks and ticketing systems:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Prometheus metrics and Grafana dashboard panels for run status and reclaimed space.<\/li>\n\n\n\n<li>Webhook notifications to ticketing (Jira, ServiceNow) and collaboration channels (Slack, MS Teams).<\/li>\n\n\n\n<li>Automatic creation of incident tickets for failed verification or archive storage issues.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Safe operations &amp; Best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Always run in dry\u2011run mode for new rules and schedule apply runs during maintenance windows.<\/li>\n\n\n\n<li>Use owner whitelists to prevent accidental cleanup of business\u2011critical tags.<\/li>\n\n\n\n<li>Test restore procedures regularly \u2014 archived data must be recoverable within SLAs.<\/li>\n\n\n\n<li>Keep multiple retention tiers and stagger purges to avoid large simultaneous restores.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Example CLI usage<\/p>\n\n\n\n<pre class=\"wp-block-preformatted has-contrast-color has-text-color has-background has-link-color wp-elements-a01db3fd3b41bf98cb9228c2740bb758\" style=\"background-color:#f6f9ff\"># Run a dry\u2011run with rule set cleanupctl scan --config \/etc\/cleanup\/config.yaml --rules \/etc\/cleanup\/rules.d\/cleanup-rules.yaml --mode dry-run --output plan-2026-02-06.json\nApply the approved plan\n\ncleanupctl apply --plan plan-2026-02-06.json --approve-token ABC123 --mode apply\nVerify archived bundle\n\ncleanupctl verify --archive s3:\/\/archives\/scada\/2026-02-06-bundle.tar.gz --checksum 9f2a...abcd\n<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Deployment &amp; Support<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Cleanup ships as a CLI and a scheduled service with YAML configuration. We provide templates for scheduled housekeeping, audit export hooks, and runbooks as part of our service plans.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">audit export hooks, and runbooks as part of our service plans.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/abilit.eu\/wp-admin\/<!-- TODO: DOWNLOAD LINK --&gt;\">Download Cleanup<\/a><a href=\"https:\/\/abilit.eu\/service\"> <\/a><a href=\"https:\/\/abilit.eu\/service\">Service Plans<\/a><a href=\"mailto:ops@abilit.eu?subject=Cleanup%20Inquiry\">Contact Ops<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Abil\u2019I.T. \u2014 Cleanup<\/strong><\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\">Download Cleanup<\/a><\/div>\n\n\n\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/abilit.eu\/service\">Service Plans<\/a><\/div>\n\n\n\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"mailto:info@abilit.eu?subject=Cleanup%20Inquiry\">Contact Ops<\/a><\/div>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Contact: <a href=\"mailto:info@abilit.eu\">info@abilit.eu<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Core capabilities Quick facts Package: Cleanup vA.B Modes: dry\u2011run, apply, schedule Download:Get package How Cleanup works Cleanup analyses database records and time\u2011series meta information using configurable rules. It generates a change plan (human\u2011readable diff) and an auditable runbook. Administrators should always review the plan in dry\u2011run mode before applying changes. Archival moves data to a [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":0,"parent":1547,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1606","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/pages\/1606","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/comments?post=1606"}],"version-history":[{"count":3,"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/pages\/1606\/revisions"}],"predecessor-version":[{"id":1609,"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/pages\/1606\/revisions\/1609"}],"up":[{"embeddable":true,"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/pages\/1547"}],"wp:attachment":[{"href":"https:\/\/abilit.eu\/index.php\/wp-json\/wp\/v2\/media?parent=1606"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}