Terms Beginning With 'b'

Big Data

  • November 03, 2020
  • Team Kalkine

What is Big Data?

Big data refers to a collection of structured, unstructured, and semi-structured data that organizations collect to mine information that can be used across various advanced analytics applications, including predictive modelling.

Big data is massive and contains multiple complex datasets that are difficult for traditional data processing software to manage. A key advantage of using big data is that the large datasets can be used to tackle complicated business problems that were challenging to address earlier.

INTERESTING READ: Big Data- The Emerging Gold of The Modern Era

What is Big Data Analytics?

Analysis of big data facilitates analysts, business users, and researchers to make fast and better decisions using untapped data. The businesses can use advanced analytics techniques to gain further insights from the earlier untouched data sources independently or together with existing enterprise data.

The advanced analytics techniques include machine learning, predictive analytics, and text analytics, among others.

What are the different forms of Big Data?

There are three types of big data. These are:

  1. Structured
  2. Unstructured
  3. Semi-Structured

Structured Data:

Structured data are those that can be stored, accesses and processes in a fixed format. An example of structured data is a student database.

Unstructured Data:

As the name suggests, unstructured data have an unknown form or structure. Further, the size of these unstructured data is enormous, and there are multiple challenges while processing such data.

An example of unstructured data is when you do a google search for a topic, say big data. When you run the search, you may get various topics, images, books, authors, different websites, and videos as results. In current times, there are vast data available in an unstructured format, and many businesses fail to derive benefits from such form of data.

Semi-Structured Data:

Semi-structured data has the qualities of both structured and unstructured data. An example of semi-structured data is an RDBMS or relational database management system. Another example would be XML or HTML code used for creating web pages.

GOOD READ: Artificial Intelligence and Big Data- The Powerhouse of a Digital Future

Characteristics of Big Data

  1. Volume: Big data, as the name suggests, refers to enormous datasets. Which particular dataset falls under the category of big data relies on the volume of data within that dataset. Hence, it is essential to keep the volume of data in mind while handling big data.
  2. Variety: Variety refers to the various sources and the kind of data. The scope of data is not restricted to spreadsheet and databases anymore but has now become more extensive. The data are now available in various formats such as emails, videos, photos, audio, and blogs.
  3. Velocity: Velocity is related to the speed at which data is created and simultaneously processed to derive meaningful output.
  4. Variability: Variability refers to the inconsistency in the data that might exist and impact the process of managing the database efficiently.

Applications of Big Data

Big data is useful for companies to interpret huge data at a faster rate which helps them to take important decisions related to the business, product development, expansion any many more.

Let us look at some of the industry where we can see the application of Big Data across sectors:

  1. Banking and Securities: In the banking and the securities industry, big data plays a critical role in detecting any fraudulent activity, enterprise credit risk reporting, analytics for trading and so on.
  2. Insurance: In the insurance industry, big data is useful in providing customer insights for transparent and simpler products. It also helps in better customer retention.
  3. Communications, Media, and Entertainment: Big data can be used for developing content based on the preference and behaviour of the viewer. Through big data, the companies from the Communications, Media and Entertainment industry can create content for its target audience, recommend related content and also measure content performance.
  4. Education: In the education industry, big data can be useful for measuring a teacher’s effectiveness to ensure that both teacher and students have a pleasant experience during the session. Data can help make teaching experience better. Further, big data in this industry can help to track the overall progress of the students as well.
  5. Healthcare Providers: Big data can help the doctors to use evidence-based medicine instead of administering several medical/lab tests to all patients visiting the hospital.
  6. Manufacturing and Natural Resources: In the manufacturing industry, Big data can be useful in solving the day-to-day challenges faced by the manufacturing companies such as supply chain, the shipment of goods, demand forecasting and many more.
  7. Retail and Wholesale industry: In this industry, big data support in optimizing staff based in the shopping pattern of the customer, local event. Like many other industries, big data also helps in reducing fraudulent activities and also helps in tracking the inventory.
  8. Energy and Utilities: Energy and Utilities sectors have now started using smart meter readers that gather data for every 15 minutes. The data can be used to analyze the consumption of utilities by the customers in a better way.
  9. Transportation: Big data play a crucial role in controlling traffic, route planning, intelligent transport system.
  10. Government: In the government sector, big data has a massive application. Through big data, the government of any country can look into the socio-economic issues, any fraudulent or illegal activities within the country, health-related issue, and the most affected regions of the country and many more. Based on the output, a relevant decision can be taken, and funds can be provided accordingly.

It is the average of the worst tolerable process that is considered acceptable. The quality level of a process is accepted if its data falls in between the range of the acceptable quality limits.

What is the Dark Web?  The dark web is one such portion of the World Wide Web which is not accessible by regular search engines. The dark web is considered a hotbed for criminal activities, and it is much more than that. Various websites exist on an encrypted network inside the dark web. Standard web browsers and programs cannot find these websites. Once inside the dark web, different sites and pages can be accessed like one does on the web. Scientists believe that the internet we see is only 4% of the entire ocean of the web, meaning the 96% consists of the "Deep and Dark Web".  The user interface used in the dark web is usually internet-based, but it utilises special software which is not part of the standard ones. There are dozens of web browsers to surf the internet, but they all work in the same way. These standard browsers use ports and protocols to request, transfer and view data on the Internet. The website you access may look familiar, but as you enter, it may be illegal or something familiar but otherwise not monitored by anyone else. Therefore, the deep web and the dark web are famous for being anonymous. Also read: Cyber Espionage Campaign: Strings that tie China, Australia and the US How to access dark web browser? In order to access a few areas which are restricted, the user may need a password and a process to follow. A special software called TOR (The Onion Router) or the Freenet has these non-standard connections. These browsers are unlike standard internet browsers and have a process to access. They allow the users to browse around the dark web and are focused on keeping the user identity anonymous. If hacked or accessed, the regular web browser can easily provide user information such as who the user is and whereabouts. Though the dark web is providing 100% anonymity, federal agencies have been successful in tracking down criminal activities on the dark web. It is often said that the person you are talking to on the dark web could either be an FBI agent or a criminal. Image: Kalkine   What happens inside the world of the dark web?  The dark web is famous for allowing sinister activities, but many users go on the dark web to access information which otherwise may not be accessible on standard internet. Such as users from extremely oppressive governments who cut access to the world for their citizens. Unfortunately, such confidential environments also provide open platforms to criminals, terrorists and other such individuals involved in illegal activities.   Hence, experts advise users to not access the dark web even out of curiosity as it is a lawless environment. There have been many incidents where innocent, curious users were trapped and forced to get involved in criminal activities or their digital devices hacked and compromised without their knowledge.  A study conducted by a University of Surrey researcher Dr Michael McGuires in 2019, Into the Web of Profit, shows that the dark web has become worse in recent times. Since 2016 of all the listings on the dark web suggested, 60% could harm companies. Everything illegal and criminal can be found on the dark web, it also has other legitimate options such as chess clubs or book clubs, but because of the anonymity, the user will not know whom he/she is interacting with. Inside the dark web, anonymity and lawless nature make the crimes which exist otherwise in our society hard to trace.  The payment procedure inside the dark web is also different from the World Wide Web. Most often, Bitcoin and Monero cryptocurrency are used for the transactions.    RELATED READ: Knock Knock! Cybercriminal at Your Doorstep   What’s the difference between the deep web and dark web? The dark web is part of the entire deep web and is hidden from regular browsing access. Most people confuse the deep web and the dark web as one entity. It is not. The deep web content includes anything hidden and restricted behind the security wall such as content which otherwise requires paywall or sign-in or blocked by the author. Content which cannot be easily accessible on regular internet such as medical records, membership websites, paid content are available on the deep web; hence it is also called Invisible Web.  No one really knows the total size of the internet, but the experts believe that the standard World Wide Web consists of only 4% internet, the deep web consists of 90% and dark web consists of 6% of the entire internet.  ALSO READ: Technology has changed the way we work amid the COVID-19 crisis: A look at in-demand technologies Image: Kalkine     Also read: It happens again, NZX being bullied by Cyber-attackers- Down for the fourth day   What kind of risk companies face due to the dark web?  The Into the Web of Profit report listed below threats various organisations around the world are facing, especially the ones who have weak or insufficient cybersecurity measures.   Malware attacks Distributed denial of service (DDoS) attacks Botnets Trojan, keyloggers, exploits  Espionage  Credentials access  Phishing  Refunds Customer data Operational data Financial data Intellectual property/ trade secrets    Also read: Cybersecurity and the Requirement of a Resilient Environment in Australia  Are there advantages and disadvantages to the dark web?  The dark web provides complete anonymity, the users get complete privacy to perform any activity, be it illegal or legal. Many countries in the world still have authoritarian regimes offering no civil rights to their people. To such oppressed lot, the dark web provides an opportunity to access news, information, data and also express their views. The dark web is also a perfect place for law agencies to map criminal activities while being undercover. It is also easy to commit gruesome crimes through the dark web as it is complicated and lawless. Criminals can easily use the dark web to compromise someone's privacy, steal data or private information or even hire someone to commit murder.  Do internet users need to be concerned about the dark web?  The simple answer is no unless the user is using the dark web. Study says that most young people visit the dark web out of curiosity. They do not want to indulge in any criminal activity but want to see how the hidden and secret world of the dark web operates. And that is where the possibility of the electronic device IP address getting hacked by other criminals to perform their criminal activities lies.  The earliest use of darknet dates back to the year 2000. Freenet was created at the University of Edinburgh based on a student research paper. Ian Clark wrote the paper in 1999 on the possibility of such an encrypted internet base. Freenet was created to oppose censorship and provide a platform for free speech. The most powerful dark web is TOR, and it was created by the United States government to have a secure encrypted communication in case of emergency and complete disaster. Even today, many law agencies are secretly active inside the world of the dark web to gain access in the criminal world and stay one step ahead.

Darvas Box system Every great trader/investor in the history of the markets had a specific method to approach the markets, which eventually led them to create a good fortune, Darvas Box system is one such method. It is a trend following strategy developed by Nicholas Darvas in the 1950s to identify stocks for good upside potential. This is one of the few methods to trade the markets which uses the combination of both the technical analysis and fundamental analysis for a much more refined decision.  The fundamentals were used to identify the stocks, and technical analysis was used to time the entry and exits. Who was Nicholas Darvas? Nicholas Darvas was arguably one of the greatest stock traders/investors during 1950s – 1960s, but surprisingly he was a ball dancer by profession and not a professional stock trader. Even while trading and building his fortune, he was on a world tour for his performances in many countries and took up trading as a part-time job. In November 1952 he was invited to a Toronto Nightclub for which he received an unusual proposition of getting paid in shares by the club owners. At that time, all he knew was there is something called stocks which moves up and down in value, that’s it. He accepted the offer and received 6k shares of a Canadian mining company Brilund at 60 cents per share, with the condition that if the stock falls below this price within six months, then the owners would make up the difference. This was the introduction of a professional ball dancer to the stock market. Nicholas Darvas couldn’t perform at the club, so he bought those shares as a gesture. Within two months, Brilund touched $1.9, and his initial investment of $3000 turned to $11400, netting in almost three times of his investment. This triggered a curiosity into the stock markets, and he started to explore trading. Origin of the Darvas Box theory Initially, he was trading on his broker’s recommendation, tips from wealthy businessmen, he even approached some advisory services or any source that he could get his hands on for the tips, but all led him to losses. After losing a lot of money, he decided to develop his own theory, and after a lot of trial and error, his observations and continuous refinements he eventually invented his theory “The Box Theory”. So what exactly is the Box Theory? Fundamentals Analysis As stated earlier, the box theory uses a judicious bend of both the technical and fundamentals. Darvas believed that in order to spot a good stock or even a multibagger, there should be something brewing up in the respective sector as a whole or some major fundamental change in that specific company. Generally, the fundamentals that Darvas used to study were on a broader sector level, and not the company-specific fundamentals. Even for the specific company Darvas used to look from a general perspective like, is the company launching a new product which could be a blockbuster hit. He completely refrained from looking at numbers and financial statements as his initial experiment with ratios and financial statements didn’t yield any good result. To know more on the three financial statements read: Income Statement (P&L) Balance Sheet Cash Flow Statement Technical Analysis Darvas was a big believer in price action and volume of the stock. He believed if some major fundamental changes were to take place in a company, this soon shows up in the stock price and its volume of trading as more people get interested in buying or selling the stock. With his observations here realized by just observing the price action, he can participate in the rally which gets triggered by some major fundamental development without actually knowing about the change. Using the box theory, Darvas used to scan stocks based on rising volume as he needed mass participation in the rally. Also, he only picked up those stocks that were already rising. His theory is all about “buy high, sell higher” instead of the conventional belief of “buy low, sell high”. After the stock satisfies both the parameters of increasing price and volume with major underlying fundamental change, Darvas looks to enter the stock. Good read on momentum trading. How and where to enter? Major part of the box theory is based on entry and exit levels. To enter a stock, Darvas looked for a consolidation phase preceded by a rally. A consolidation phase is the price action wherein the price moves up and down in a tight range, that is, a non-directional move. He would then mark the high and low of the consolidation phase with the horizontal line, essentially making it a box-like structure, hence the name “Box Theory”. The high point is called the ceiling, and low is called the floor. Whenever the stocks break above the ceiling, Darvas would look to buy one tick above the ceiling with one tick below floor as a stop-loss point. Pyramiding Darvas discovered early on, in order to become successful in the market your winning bets should yield much more profit than the loss in the losing bets. This led him to do pyramiding in his winning trade, which is clearly defined in the box theory. Pyramiding means to increase the existing position if the stock is going in the favour, which leads to a much higher profit in the winning trades. According to the box theory, the repetition of the entry criterion is the new signal for adding onto the existing position. In other words, after a position, if the stocks stage the same setup, that is, a consolidation after a rally, then the break above the ceiling of this new box would signal to increase position with the revised stop loss of 1 tick below the new floor. In any case, whenever the stock falls below the current floor, the entire position would we sold off at once. This is the only exit condition in the box theory, and there is no method of booking profit upfront as Darvas believed in holding on to a rising stock. The only way to book profit is to let the stock to take out the revised stop loss.

What is Data Analytics?  Data Analytics involves a set of quantitative and qualitative approaches and processes that can be used to determine useful information for business decision-making. The process involves various patterns and techniques, including: extracting a raw database, and categorising it to identify and analyse the behaviour, relation and connection of the results.  The ultimate goal is to acquire valuable information in order to make decisions for businesses’ benefit and productivity.  In today's competitive times, most companies chalk out their business plan with the help of data analytics. With organisations becoming customer-service oriented, data analytics has become a critical tool to reach the target audience in an effective manner while understanding their requirements. Once data is collected, it is analysed and stored according to organisations’ requirements.  The data analysis process has multiple layers involved, and its diverse modules are not just used in businesses but also in science and social science fields. Rather than making decisions based on just available information, one can utilise data analytics in examining the data in standard ways and churning out the results from it.  It has been observed that companies generally make decisions based on past references and future outcomes. Data analytics appears advantageous in providing useful information towards this end.  Why do Businesses Need to Use Data Analytics?  Many data analytics’ tools and softwares are readily available these days. These systems use resources, such as machine learning algorithms and automation.   Data scientists and analysts are counted amongst the leading career options as well. These professionals use data analytics techniques while researching and presenting useful information for businesses to increase productivity and gain. The process helps companies understand their target audience and determine effective ways to cater to their needs. Data analytics can further be used to design strategies in marketing campaigns and promotions and also evaluate its results.  Data analytics is primarily used in business-to-consumer (B2C) processes to boost business performance and improve the bottom line. There are data collection firms which gather consumer information and provide it to the businesses so that the companies can effectively influence the market. The collected data is not only used to understand and impact consumer behaviour but also determine market economics and its practical implementation.  The data used in the process can be either be data collected in the past or newly updated data. There are various methods to manage consumer and market information. It may come directly from the customers or potential customers or can be purchased from the data collection vendors. The data primarily includes audience demographics, behavioural patterns and expense threshold.  How Can Data Analytics be Effectively Used in Business Processes? Data analytics is an ever-evolving technique. Earlier, the data was collected manually, but with the rise of internet and technology, data is now collected online with the help of search engines and social media platforms. Subsequently, the information is analysed through available software.  Here is a list of some key steps businesses can follow to leverage the benefits of data analytics: Set up crucial metrics: This step reduces the guesswork and provide data-based insights to the businesses. Before embarking on the data analytics process, it is vital to determine the goal for your business. Analysing customer data helps in understanding conversion rate, consumer spending ability, demographics etc. The results of the analysis can support the businesses while making decisions in launching an advertising or marketing campaign. Similarly, the unwanted data can be erased from the database so that the brands can focus on their right target audience. The relevant metrics will change the course of the company and push it in the right direction. Moreover, once your key metrics are set, even when the market conditions change in the future, you can adjust the metrics according to the requirement and achieve the results. Set a clear module: It is important to examine the data correctly by avoiding common mistakes. An ambiguous path can produce confusing insights while wasting time and energy of businesses.  Therefore, it is recommended to draw a clear goal in order to achieve actionable insights. The data, when collected from different sources, need to be merged accurately in the analytics model. Businesses can modulate their data analytics systems either manually or through automation. There are various data modelling practices available in the market. The best use of these techniques can simplify the process of modelling complex data.  Data visualisation: Once the relevant data is collected, and the modules are set to analysis, visualisation of that data will assist in understanding the information correctly. When the businesses have an acute knowledge of what their target audience wants, they can then focus on strategising advertisement and content, which matches the consumers' interest.  It is the critical step in the data analytics process to distinguish insights from information.  Not everyone is comfortable dealing with numbers. Hence, ensuring that key stakeholders understand essential points and information can be displayed in a visually appealing format seem crucial to capitalise on data effectively. Right tools to implement insights:Having access to data and insights can get overwhelming. However, the information is worthless if the businesses are unable to implement it successfully. While it is important to collect the data and set critical metrics and modules to analyse it, it is also imperative to translate the data into practical actions. The eventual goal is to improve sales or grow profits. It is ultimately in the marketers' hands to transform the gained insights into a successful implementation. The consumers' insights should be incorporated while establishing a marketing plan and at all decision-making steps. 

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it. OK