Using Gridline
Quality rules
Say once what good data means for your company, and every upload is checked against it.
A rule is something you decide good data must satisfy. Write it once; Gridline checks every upload against all your rules and tells you which failed. This is what turns a report from a description into a verdict.
The eight kinds of rule
| Kind | In plain words | For example |
|---|---|---|
| Column is required | The file must have this column. | Every orders file has an order_id column. |
| Most empty cells | At most this share of the column may be empty. | email is at most 5% empty. |
| Column type | The column must be this kind of data: whole numbers, numbers, yes / no, dates or text. You can allow a share of values to disagree (none, by default). | amount is numbers. |
| Lowest value | No number may be below this. | quantity is never below 0. |
| Highest value | No number may be above this. | discount is never above 100. |
| Values are unique | No value in the column may repeat. | order_id never repeats. |
| Most repeated rows | At most this many whole rows may repeat. This one is about the whole file, so it needs no column. | No repeated rows at all. |
| No personal data | The file must not hold personal or secret data: email addresses, phone numbers, card numbers, bank account numbers (IBAN), IP addresses, keys and tokens, or dates of birth. You can look for any of them or for just one kind. This one is about the whole file, so it needs no column. | No file with card numbers in it is ever shared. |
Rules look at a file's statistics, never its rows. They are about the shape and health of the data, so checking them never needs to read cell values. No personal data works from the same statistics: the report keeps only what kind of data a column looks like, never a value.
Add a rule
Only admins can add, change or delete rules. Everyone in the company can read them, so an employee knows what an upload is checked against.
Open Quality rules
Choose Quality rules in the sidebar.
Choose Add a rule
It is at the top right of the list, next to how many rules your plan allows, for example 3 of 25 rules on your plan.
Name it
Pick a name that will read well in a report, such as “Emails are filled in”. Names can be up to 80 characters.
Pick what to check
Under What to check, choose one of the seven kinds above. This cannot be changed later: to use another kind, delete the rule and add a new one.
Say which column
Type the column's name as it appears in the file's header. Capitals do not matter. Most repeated rows is about the whole file, so it has no column.
Set the limit
Each kind asks for its own number, with a hint under it. For example, 5 for “at most 5% empty”.
Choose how serious a failure is
Error or Warning: see the next section. Error is the default.
Add the rule
Choose Add rule. It applies to every file uploaded from now on. Files already uploaded keep their old results until you choose Check again on their report.
Each rule in the list has three buttons: Turn off (or Turn on), Edit and Delete. A rule that is turned off is kept but not checked, and is marked Off. Editing shows the rule's kind but does not let you change it. Deleting asks you to confirm, and reports already built keep their result for the deleted rule.
Errors and warnings
| Counts in the score | When it fails | |
|---|---|---|
| Error | Twice | Tells the uploader and every admin. |
| Warning | Once | Only lowers the score. Nobody is notified. |
The notification names the rules that failed and the score. It never contains a value from the file.
When a rule does not apply
Your rules cover every upload, but your files are different from each other. A file with no amount column is not wrong for an amount rule, so such a rule is skipped for that file, not failed. A skipped rule counts for nothing in the score.
Limits
| Plan | Rules you can keep |
|---|---|
| Free | 3 |
| Basic | 25 |
| Premium | No limit |
Disabled rules count towards the limit. On any plan you can have at most 10 “values are unique” rules, because each is checked by remembering the values of its column. Going past a limit is refused with the reason. A downgrade that would leave you over the new plan's limit is refused until you delete some.
From code
Creating, listing, changing and deleting rules are available over HTTP, with every setting spelled out: see Quality rules API.