Clear Web, Deep Web and Dark Web

An Introduction

All of you might have heard about the Internet and WWW, but today we’re about to examine the other side of the internet or how much do we know about the internet? Yes, as we always say there are two sides to a coin, there exists a dark side for the internet too.

Before going too deep into the WWW, we’ll see how to distinguish between the Clear, Deep and Dark webs. The illustration below can put some light on the things we are discussing here, ##

  • Clear Web: This is the most common one. It’s the Internet we use on a daily basis either on mobiles or on Desktops/Laptops to check emails, to read news, to access Facebook, twitter, Instagram etc and for online shopping, booking tickets which we browse regularly.
  • Deep Web: It is the part or a subset of the Internet that isn’t necessarily malicious, but is simply too large and/or obscure to be indexed due to the limitations of crawling and indexing software (like Google/Bing/Baidu).  This means that you have to visit those places directly instead of being able to search for them. So there aren’t directions to get there, but they’re waiting if you have an address.
  • Dark Web: The Dark Web (also called Darknet) is the ill-famed subset of the Deep Web that is not only not indexed, but also is used by those who are purposely trying to control access because they have a strong desire for privacy, or because what they’re doing is illegal. The Dark Web often sits on top of additional sub-networks, such as Tor, I2P, and Freenet, and is often associated with a criminal activity of various degrees, including buying and selling drugs, pornography, gambling, etc. Though the deep web makes up 95% of all the internet the dark web only consist of about .03%. But that small section has millions of monthly users. The dark web is usually what people actually mean when they refer to the deep web. Although both are technically correct it is important to keep it separated so there is a standardised phrase.

Deep Web & Dark Web

Computer scientist Mike Bergman is credited with coining these term in 2000. The Dark web is a small part of the Deep Web which is not indexed by search engines. Sometimes the term “deep web” is mistakenly used to refer specifically to the dark web. Most of the web’s information is buried far down  on  sites, and standard search engines do not find it. Traditional  search  engines like Google,Bing etc cannot see or retrieve content in the deep web. The  portion of  the web that is indexed by standard search engines is known  as the  surface web or clear net or clear web. As of 2001, the deep web was several orders of  magnitude larger than the surface  web.

Methods   which prevent web pages from being indexed by traditional search   engines may be categorized as one or more of the following: **
  • Dynamic content:  dynamic pages  which are returned in response to a submitted query or  accessed only  through a form, especially if open-domain input elements  (such as text  fields) are used; such fields are hard to navigate  without domain  knowledge.
  • Unlinked content: pages which  are not linked to by other pages, which may prevent web crawling  programs from accessing the content. This content is referred to as  pages without backlinks (also known as in links). Also, search engines do  not always detect all backlinks from searched web pages.
  • Private Web: sites that require registration and login (password-protected resources).
  • Contextual Web:   pages with content varying for different access contexts (e.g., ranges   of client IP addresses or previous navigation sequence).
  • Limited access content:  sites that limit access to their pages in a technical way (e.g., using  the Robots Exclusion Standard or CAPTCHAs, or no-store directive which  prohibit search engines from browsing them and creating cached copies).
  • Scripted content:  pages that are only accessible through links produced by JavaScript as  well as content dynamically downloaded from Web servers via Flash or  Ajax solutions.
  • Non-HTML/text content: textual content encoded in multimedia (image or video) files or specific file formats not handled by search engines.
  • Software:  certain content is intentionally hidden from the regular internet,  accessible only with special software, such as Tor, I2P, or other  darknet software. For example, Tor allows users to access websites using  the .onion host suffix anonymously, hiding their IP address.
  • Web archives:  Web archival services such as the Wayback Machine  enable users to see  archived versions of web pages across time,  including websites which  have become inaccessible, and are not indexed  by search engines such as  Google.

How to access

Now we will discuss the most important part, Ho does we access the Deep web and Dark Web?
If you have an email id, a facebook account, a twitter account or collectively saying any of these social media accounts or if you are a person using internet banking  you are accessing the Deep web on a daily basis.
Wondering how.!! The databases which store your user information and other details about your account reside in the deep web.
So, now the sole enigma is on the access to Dark Web.
You cannot simply access the dark web from a normal web browser like Chrome, you can only access the dark web through a dark web browser. The most famous of these deep web browsers is called Tor and this is the one we recommend you get if you’re looking to get onto the deep web.

Tor

Tor is free software for enabling anonymous communication. The name is derived from an acronym for the original software project name “The Onion Router”. Tor directs Internet traffic through a free, worldwide, volunteer network consisting of more than seven thousand relays to conceal a user’s location and usage from anyone conducting network surveillance or traffic analysis. Using Tor makes it more difficult for Internet activity to be traced back to the user: this includes “visits to Web sites, online posts, instant messages, and other communication forms”. Tor’s use is intended to protect the personal privacy of users, as well as their freedom and ability to conduct confidential communication by keeping their Internet activities from being monitored.
It was developed in the mid-1990s by United States Naval Research Laboratory employees, mathematician Paul Syverson and computer scientists Michael G. Reed and David Goldschlag, with the purpose of protecting U.S. intelligence communications online. Onion routing was further developed by DARPA in 1997
Downloads of Tor soared in August by almost 100% as the general population became more and more concerned about their privacy amid revelations about US and UK intelligence agencies monitoring web traffic. In short, more and more people are turning to the deep web to get their internet fix and protect their information.
This is because when you’re using Tor – or any other deep web browser – you are truly anonymous and your location cannot be picked up and neither can your browsing habits. Essentially nothing you do in the deep web can be monitored and as such the deep web is becoming a more attractive option for all internet users – those who know about it at least.

.onion

.onion is a special-use top level domain suffix designating an anonymous hidden service reachable via the Tor network. Such addresses are not actual DNS names, and the .onion TLD is not in the Internet DNS root, but with the appropriate proxy software installed, Internet programs such as web browsers can access sites with .onion addresses by sending the request through the network of Tor servers. The purpose of using such a system is to make both the information provider and the person accessing the information more difficult to trace, whether by one another, by an intermediate network host, or by an outsider. Addresses in the .onion TLD are generally opaque, non-mnemonic, 16-character alpha-semi-numeric hashes which are automatically generated based on a public key when a hidden service is configured. These 16-character hashes can be made up of any letter of the alphabet, and decimal digits from 2 to 7, thus representing an 80-bit number in base32. The “onion” name refers to onion routing, the technique used by Tor to achieve a degree of anonymity.
Apologies for deviating from the topic of accessing the Dark Web, Without a mention about TOR and .onion we cannot jump to access the Dark Web.

Steps to accessing the Dark Web

The method mentioned below is the simplest way to access Dark Web. There are other methods and TOR specific OS to access the contents of Dark Web
Download and install TOR Browser Bundle from the TOR project site.
TOR Browser Bundle is a browser configured to browse using the TOR relay network.  The browser component is actually built using the Firefox codebase, so if you have used Firefox you will find it pretty familiar.  It comes properly configured and won’t install any malware or cruft on your computer, as it is written and maintained by freedom fighters, not some large corporation.
https://www.torproject.org/projects/torbrowser.html.en 
Once you install it, fire up your tor browser and you are good to go into the Dark Web.
You can browse on to the Dark Web sites using this browser by passing the links in the address bar of the browser. The following link will give you a good start. Don’t forget to go through the links “The Matrix” and “How to Exit the Matrix” in this link. Also, I personally recommend using Linux or any other Non-Windows for playing with TOR and Dark Web. Please check this link for taking some security measures Do’s and Dont’s

http://zqktlwi4fecvo6ri.onion/wiki/index.php/Main_Page

Search Engines TOR specific:

More onion URLs are found in the  Hidden Wiki

## – This illustration is just for demonstration purpose

** – This content is as is fromWikipedia

Disclaimer:

This is a personal blog. All content provided on this blog is for informational purposes only. They are collated from different sources and some are my own.The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site.

The owner of THIS BLOG will not be liable for any errors or omissions in this information nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of this information 
Data courtesy : Rational WikiPopular Science, Wired.com, TORProject, Hacked