Enterprises use data virtualization software such as TIBCO Data Virtualization software to reduce data bottlenecks so more insights can be delivered for better business outcomes.
For developers, data virtualization allows applications to access and use data without needing to know its technical details, such as how it is formatted or where it is physically located.
For developers, data virtualization helps rapidly create reusable data services that access and transform data and deliver data analytics with even heavy-lifting reads completed quickly, securely, and with high performance. These data services can then be coalesced into a common data layer that can support a wide range of analytic and applications use cases.
Applications Developers Not Yet Benefiting
On the other hand, data virtualization adoption by application developers lags their more analytics oriented peers. That’s interesting because if you are an application developer, you know that nearly every application you build requires multiple reads. Some are simple queries, for example, looking up an address in a shipping application. But others are more difficult, for example when you need to provide pricing for an order entry application you may need to query customer lifetime value, current and prior quotes, current promotions, and other data to calculate what to charge the customer for this particular purchase.
Regardless of use case, every read transaction you build requires you to:
• Understand and access multiple data sources, perhaps
even combine them
• Transform that data from its native IT structures and syntax to
a format and syntax your application can readily consume
• Synchronize the delivery of the data from the source(s) to
your application when requested
Automating these functions for developers is exactly what data virtualization was built for. So where is the disconnect? What should applications developers know about data virtualization that their analytics developer and data engineering colleagues have already figured out?
1. Simplify Understanding and Accessing Diverse Data As an application developer you need to query data from diverse sources. These queries typically include:
• Traditional sources such as relational databases
• Various NoSQL sources including Hadoop, graph, keyvalue, and other databases
• Cloud sources including SaaS applications such as
Salesforce.com and cloud providers such as Amazon
Redshift and Microsoft Azure
• IoT device
For each source, you need to understand the structure, relationships, access methods, and more. And frankly, that is hard because your primary focus is making your application work, not figuring out the darn data. Data virtualization middleware provides standard data source connectors to all the most popular data sources, so you can use a common tool and approach all these sources. Further, data virtualization discovery tools introspect source data and uncover hidden relationships automatically for you. So understanding your data is also less of a challenge
2. Federate Data with High Performance
Every application developer knows that simple, single-row reads from a single table perform well. But when you need to query larger datasets from multiple sources, combine them, and serve them to the application, performance can often suffer. When you use data virtualization, you don’t have to worry as much about fine-tuning your queries to achieve the response times your application requires. Advanced query optimization algorithms and techniques—such as cost- and rule-based optimization, pushdown, and automatic query re-writing to use the most efficient join strategy, and more—are but a few of the capabilities data virtualization products employ to ensure high performance.
3. Leverage Powerful Tools for Complex Data Transformations
Sometimes you get lucky and both your source and consumer are relational and you can use simple SQL statements and ODBC to query and deliver the data. But how do you do the transformation required when the source is a relational database and the consumer is a RESTFUL application? Or when the source is an XML document and the consumer is a client-server application? And even more challenging is doing this when your application requires a combination of source types and more complicated transformations. TIBCO Data Virtualization software lets you easily build complex transformations graphically or via any of five languages including SQL, SQL script, XQuery, XSLT, and Java functions. You can even mix and match techniques within a single data delivery service.
4. Reuse Analytic Data Services for ApplicationsUse Cases
Have you ever built a data service and then found out later that one already existed? Is there anything more frustrating? If your colleagues over in analytics development are already using data virtualization, you can easily take advantage of data services they have already built. In fact, for some of the larger data virtualization customer deployments, data engineers are seeing as much as 80% reuse. Would even half of that be nice? Let’s consider the case where an existing analytic data service build with data virtualization is pretty close to meeting your needs. You can also use data virtualization to modify and extend an existing service. One example might be enabling an existing ADO.net data delivery service to also be called via SOAP or REST. Another might be to provide an additional filter on the query or an additional validation step. With data virtualization, you can do this kind of fine tuning in minutes, and cut days off your application time to solution.
5. Provide Enterprise-wide Data Services for Analytics and Applications
Both applications and analytics require data; Lots of it from lots of places, And demand is rising. So doesn’t it make sense that application and analytics developers would team up to address this challenge? Reference data services used frequently for lookups is an example of a low-hangingfruit opportunity—While standard, enterprise data services conformed to an enterprise data model is a larger, yet achievable, aspiration.
With data virtualization products such as TIBCO Data Virtualization software, you can provide secure, reusable, highly performant data services that can support the needs of hundreds of developers across both analytics and applications use cases. Further, adding an API Management tool such as TIBCO Mashery software will enable these data services to be easily shared across the enterprise, as well as externally with customers, suppliers, and others. In addition, this combination lets companies adopt a more unified approach to the complete application and analytics API development lifecycle from design through retirement, and thus gain the agility, efficiency, and control that naturally results.
CONCLUSION
While data virtualization has been a powerful force for analytics developers and data engineers, application developers have yet to take advantage—Yet the challenges faced and advantages delivered are similar and can be shared. This paper provides application developers with five reasons to
consider using data virtualization
