GENERATIVE artificial intelligence (GenAI) is one of the most exciting technologies to have hit enterprises in recent years. Its use is on the upswing with many businesses in Malaysia and throughout the region having embedded it into their operations to drive innovation and growth.
Just in May 2024, Prime Minister (PM) Anwar Ibrahim stressed that Malaysia should move faster to build a robust ecosystem to drive the development and adoption of AI technologies.
Speaking at the 2024 Malaysia Artificial Intelligence Nexus event, the PM outlined the urgency of producing more skilled engineers to meet industry demands.
This included the introduction of four core areas of Malaysia’s AI ecosystem, namely (i) Governance to ensure a systematic AI ecosystem; (ii) Catalysation through the Malaysia Artificial Intelligence Consortium (MAIC); (iii) Industry collaboration; and (iv) Talent development through education and training programmes.
Looking further ahead, the PM also called for a renewed focus on engaging young people in science, technology, engineering and mathematics (STEM) subjects which are crucial for building a sustainable AI ecosystem.
Huge potential
The Malaysian GenAI market is indeed set to grow by leaps and bounds. According to Statista, the local market is projected to reach US$179.50 mil in 2024. The market size is expected to show a compound annual growth rate (CAGR 2024-2030) of 46.48%, resulting in a market volume of US$1.7 bil by 2030.
Indeed, the impact of GenAI is poised to revolutionise industries, driving unprecedented innovation and economic growth in Malaysia.
Yet the starting point of all these varied uses is data. The underlying language learning model (LLM) run on a large amount of data which must be well-organised to function effectively.
Think of data and GenAI as like the interlocking pieces of a Mondrian puzzle, each segment a crucial part of the larger picture of business innovation. To lay a strong foundation that allows firms to maximise ROI (return on investment) from their AI investments, corporate data must be meticulously prepared and optimally used with the right tools throughout its life cycle.
The 5W-1H rule
In many organisations, the most common sources of data are internal product documentation, history of customer purchases and support information, and media such as videos and images. For the GenAI engine to efficiently use this wide range of data, proper organisation is paramount.
Enter the 5W-1H way of preparing data:
- When
It’s important to know when and how often data is collected. In-house product documentation is created as product releases and updates occur.
Customer information is stored in CRM (customer relationship management) or other systems as needed. In the case of video and audio, the files may be generated in real time. A system must be built to promptly collect data when they are created.
- Where
Data exists in many places. In-house product documentation is edited on a local PC (personal computer) and can be stored on a file server or online in the cloud. Customer information is typically stored in a database on-premises or in the cloud.
Video and audio are often generated at the edge and must be collected over the network. A system needs to be properly set up to collect data at each location.
- Who
It’s crucial for ownership of the various data to be clearly defined. Only then can business owners and stakeholders accountably work with the owners to better manage and protect their data, and ensure consistent data usage.
- What
Data comes in many forms and formats. Product documents are typically stored in office file formats such as Word and Excel while customer information is often stored as structured data in databases.
Media files which are unstructured include video and audio. Understanding the type of data being handled enables organisations to pre-process and analyse them effectively.
- Why
When tapping data for AI, issues to be solved must be defined from the start. Doing so enables an organisation to zoom in on the most relevant data. Measurable numerical targets should also be identified and used to track progress over time.
- How
Appropriate methods need to be used for data collection, taking into account the nature and location of data. For instance, the collection of file server data uses protocols like NFS (network file system) or CIFS (common internet file system).
Data collection from databases uses appropriate accounts and database-specific protocols. And for collection of real-time data, the ability to work with edge devices is vital.
Boosting speed to deployment
After organising your data, the next step is to accelerate the deployment of your GenAI system. MLOps (machine learning operations) is key to streamlining workflows and speeding up the shift from AI development to production.
At this stage, it’s essential to fully utilise your organisation’s storage infrastructure, especially for MLOps and data operations including data pipelines and DataOps (data operations).
To facilitate a smooth and rapid deployment, the following key features of enterprise storage systems should be leveraged:
- Optimising Enterprise Data Management
Enterprise storage facilitates the collection of data from diverse sources through multi-protocol support, including NFS and CIFS, streamlining essential management tasks such as data protection, versioning and security for AI applications.
Moreover, recent advancements in container support enable MLOps and DataOps, thus allowing data scientists to concentrate on AI model development.
- Advancing Hybrid Multi-Cloud Strategy
Setting up a hybrid multi-cloud environment and data mobility is crucial, especially for GenAI and LLMs (large language models) that rely on cloud-exclusive services and functionalities.
Establishing a data pipeline between on-premise environments and the cloud offers flexibility and scalability to support AI initiatives. Enterprise storage provides features that work with cloud vendors’ object storage or enable mirroring and caching within the cloud, hence facilitating a tailored hybrid cloud strategy for enterprises.
- Implementing Security Measures
Security in AI is vital as data is constantly exposed to the risk of cyberattacks. With increasing regulations, enterprise storage systems are equipped with security features like multi-tenancy and encryption that ensure data safety while managing it efficiently.
This allows companies to hold multiple datasets in minimal space for auditing and compliance purposes. For example, NetApp is one of the first to integrate AI and ML (machine learning) directly into enterprise primary storage to fight ransomware in real-time.
Understanding the nature of data and applying these security measures enables organisations to drive transformation and deepen business insights.
In the intricate tapestry of business evolution, data are like pieces of the Mondrian puzzle where each piece holds the potential for transformative insights. However, Malaysian organisations must first understand the nature of data that they wish to leverage.
When data is varied and widely dispersed, the collection mechanism becomes more complex. Approaches such as MLOps and data pipelines as well as DataOps can help from the AI-centric and data-centric operational perspectives respectively.
Additionally, deploying an intelligent data infrastructure will help empowers organisations to unlock the value of their data with AI. This is why NetApp continues its commitment to innovation with unified data storage systems to help organisations achieve robustness for the most demanding workloads.
Appropriately blending the approaches along with a powerful, intelligent data infrastructure will accelerate generative AI programmes in Malaysia, thus bringing business benefits sooner rather than later. – June 12, 2024
Alwyn David is the country manager of NetApp Malaysia.
The views expressed are solely of the author and do not necessarily reflect those of Focus Malaysia.
