Welcome!

Microservices Expo Authors: Pat Romanski, Elizabeth White, Liz McMillan, Yeshim Deniz, Zakia Bouachraoui

Related Topics: @DXWorldExpo, Microservices Expo, Microsoft Cloud, Containers Expo Blog, @CloudExpo, Apache

@DXWorldExpo: Blog Feed Post

Big Data On-ramp

Here are five areas of consideration for Big Data on-ramp: Structure, Location, Objective, Participant, and Event (SLOPE)

Due to the unprecedented volume, variety, and velocity of Big Data, it is neither trivial nor straightforward to find a clear path to jumpstart the Big Data journey. This space is overwhelmingly crowded with so many immature options and evolving solutions. To some extent it is somewhat confusing and daunting. Where can you find an entry point? What is the most effective way to get on board? Which aspects should you be mindful of? How can you not miss the paramount things?
Why do you need to begin with the basics?

Here are five areas of consideration for Big Data on-ramp: Structure, Location, Objective, Participant, and Event (SLOPE).

  • Structure: The data format is the first and foremost factor. Confining to the traditional structured contents is no longer sufficient in this era. We have to pay close attention to how we will deal with the unstructured and semi-structured information that will be imported and analyzed in the short term and long run.
  • Location: Where data reside and how they move around inside or outside an enterprise have an influential impact on the overall Big Data ecosystem. A sophisticated messaging platform should be employed in a complex environment entailing heterogeneous data sources and consumption. The data locality is also important for distributed processing with a viable hosting model.<
  • Objective: The reasonable goal and right level of expectations should be set up at the very beginning to develop solid business cases. Big Data as a whole is not just for the sake of moving to the Big Data technology. Rather, it is an advanced discipline to transform a problematic environment to a realistic target state. For example, eliminating data silos is a must, but it also brings pains and conflicts during the execution.
  • Participant: It is crucial to conceive Big Data solutions from a user-centric perspective. All stakeholders involved need think coherently about the value chain of the data as assets. The priorities and preferences among the end-users, partners, data feeders, brokers, etc. must be balanced and harmonized. A RACI or SCARI matrix should be established to specify roles and responsibilities in the governance.
  • Event: The types of interactions and access dictate what Big Data platform are the most suitable candidates for both transactional and analytical processing. Large data streaming is a feasible option to enable near real-time analytics in the scenarios like fraud detection. Quantifiable thresholds ought to be defined to explicitly describe how real is real-time.


For more information, please contact Tony Shan ([email protected]).

Read the original blog entry...

More Stories By Tony Shan

Tony Shan works as a senior consultant, advisor at a global applications and infrastructure solutions firm helping clients realize the greatest value from their IT. Shan is a renowned thought leader and technology visionary with a number of years of field experience and guru-level expertise on cloud computing, Big Data, Hadoop, NoSQL, social, mobile, SOA, BI, technology strategy, IT roadmapping, systems design, architecture engineering, portfolio rationalization, product development, asset management, strategic planning, process standardization, and Web 2.0. He has directed the lifecycle R&D and buildout of large-scale award-winning distributed systems on diverse platforms in Fortune 100 companies and public sector like IBM, Bank of America, Wells Fargo, Cisco, Honeywell, Abbott, etc.

Shan is an inventive expert with a proven track record of influential innovations such as Cloud Engineering. He has authored dozens of top-notch technical papers on next-generation technologies and over ten books that won multiple awards. He is a frequent keynote speaker and Chair/Panel/Advisor/Judge/Organizing Committee in prominent conferences/workshops, an editor/editorial advisory board member of IT research journals/books, and a founder of several user groups, forums, and centers of excellence (CoE).

Microservices Articles
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and value...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Consumer-driven contracts are an essential part of a mature microservice testing portfolio enabling independent service deployments. In this presentation we'll provide an overview of the tools, patterns and pain points we've seen when implementing contract testing in large development organizations.
Enterprise architects are increasingly adopting multi-cloud strategies as they seek to utilize existing data center assets, leverage the advantages of cloud computing and avoid cloud vendor lock-in. This requires a globally aware traffic management strategy that can monitor infrastructure health across data centers and end-user experience globally, while responding to control changes and system specification at the speed of today’s DevOps teams. In his session at 20th Cloud Expo, Josh Gray, Chie...