Visual Data Manipulation (VDM) differs from traditional methods such as spreadsheets or programming languages by allowing you to visually choose, configure, and combine transformations and calculations that act upon your data. While spreadsheets certainly have a visual component, they focus on showing the data itself, rather than the operations. Programming languages like Python, R, or Matlab hide even more, leaving you with just the input and output data as well as the code itself (if you are a programmer).
Simple for Beginners
Full-blown programming languages like R or Python are incredibly powerful for data work, but the critical drawback for all of them is that in order to use that power, you must be an experienced user. That’s no problem if you have a team of programmers, but what about the actual data consumers who aren’t computer scientists? They’re often stuck using spreadsheets because it’s just not practical to learn a programming language.
VDM splits apart the data and the operations so that you can focus on the flow of your data rather than on deciphering opaque algorithms or learning complicated syntax. This makes designing your model much closer to drawing a simple flowchart — something programmers often do anyway prior to coding. Not only is a flowchart natural and intuitive for most people, but it lets you build your model one piece at a time even if you aren’t certain of all the steps at the start.
Since VDM tools provide sets of nodes which each do a single, simple operation, building the steps in your flowchart only requires that you understand those simple operations. From there, it’s just a question of connecting them together to get the result you need.
Models Mirror Logic
The ability to logically create a model step by step the way you think about it is arguably the most important part of VDM. It means that mistakes are less likely and easier to fix, the time required to get something done is reduced, and the learning curve is gentle.
If you’ve used spreadsheet for very long, you’re familiar with “Excel Hell.” Workbooks become so complicated and patched together that deciphering logic, let alone errors, is next to impossible. Tracing dependencies can take huge amounts of time since the only real way to do it is to trace each affected cell back to its source. In VDM, connections between nodes show dependencies at a glance and they also show the data at that specific stage in the process. If your result is wrong, it’s a simple matter of following the data until an error is uncovered.
It has been estimated that “…88 percent of all spreadsheets have errors in them, while 50 percent of spreadsheets used by large companies have material defects”(1) and that one in five large companies have suffered financial losses due to errors in spreadsheets (2).
The Model as a Report
Once your flowchart-style model is built, it becomes both a fully functioning data process as well as a useful tool for explaining that process to others. The report is intrinsic to the model. Not only is the end result shown, but all of the steps and the results of those steps are clearly visible.
There is no substitute for traditional reporting – graphs, charts, dashboards, etc. And while most VDM tools (Transdata included) can send data to a reporting tool for presentation, those tools have a critical flaw: they only show the end result dataset(s). All of the steps to arrive at that result are lost, or at best, obscured.
With a standard dashboard, you don’t get much in the way of an explanation of where the data came from. How did you arrive at the results shown? What logic went into the process? All too often, these explanations are left to a supplemental discussion, or, perhaps, a Powerpoint presentation. VDM combines the data and the explanation in one intuitive model so that the process is the story.