How to instruct Web crawlers (search engine spiders) what to do when they visit your site. You can specify that some parts of your site should be private and non-searchable. You can control how Web crawlers index your site at different levels - the entire site, specific directories, and individual pages.

Does it seem like there are more spiders around the house right now? And do the spiders you spot seem bigger than those you've seen in recent months? Spider expert, Dr Sara Goodacre, is here with some arachnid answers.

Join citizen science efforts to monitor bats, track Joro spiders and crowd-source cemetery stories.

Above is an image of “dry ice spiders” on Mars. Every spring the Sun warms up the Martian south polar icecap and causes jets of carbon-dioxide gas to erupt through the icecap. These jets carrying dark sand into the air and spraying it for hundreds of feet around each jet forming these wonderful spider-like structures....

How do spiders make such intricate webs? Don’t get too tangled up about it. In this one-minute video, our Ask Smithsonian host, Eric Schulze, weaves his way to the answer.

Male fireflies caught in an orb-weaver spider’s web start flashing in an unusual pattern, a deadly deception that seems to attract additional victims for the spider

Unlike related species that live in forests, Tenkana spiders prefer drier areas and ground habitats and have been found in Tamil Nadu, Puducherry, Karnataka, Telangana and Andhra Pradesh 

“Spinning under the influence” is one way to describe recent activities in the Costa Rican laboratory of Smithsonian scientist William Eberhard. An entomologist at the Smithsonian’s Tropical Research Institute, Eberhard recently carried out a series of experiments in which he observed the web-building behavior of dozens of spiders under the influence of drugs—specifically, a chemical injected into their bodies by parasitic wasps.

The post Drugged spiders’ web spinning may hold keys to understanding animal behavior appeared first on Smithsonian Insider.

When it comes to brains, is bigger better? Can the tiny brain of a newly hatched spiderling handle problems as adeptly as the brain of a larger adult spider?

The post Tiny, new brains prove just as adept as large, mature brains among tropical orb-web spiders appeared first on Smithsonian Insider.

A roundup of recent articles featuring spiders, bats and rats....

The post A Halloween roundup featuring recent articles on spiders, bats and rats appeared first on Smithsonian Insider.

A new study by a team of scientists from the Smithsonian’s National Museum of Natural History, the National University of Singapore and the Slovenian Academy of Sciences and Arts have unlocked the secret to mate binding in orb web spiders, and revealed just how it calms the cannibalistic female spider.

The post Suitor’s gentle massage soothes aggressive, cannibalistic female spiders, researchers find appeared first on Smithsonian Insider.

New research on tiny spiders has revealed that their brains are so large that they fill their body cavities and overflow into their legs, say a team of scientists at the Smithsonian Tropical Research Institute in Panama.

The post Brains of tiny spiders fill their body cavities and legs, Smithsonian researchers discover appeared first on Smithsonian Insider.

Researchers at the Smithsonian Tropical Research Institute and University of Costa Rica studying why spiders do not stick to their own sticky webs have discovered that a spider's legs are protected by a covering of branching hairs and by a non-stick chemical coating. Their results are published online in the journal, Naturwissenschaften.

The post Fancy footwork and non-stick leg coating helps spiders not stick to their own webs appeared first on Smithsonian Insider.

The scientists made the spiders exercise by irritating them with a small paint brush and causing them to move around until they became exhausted. Spiders from the group with palps removed were able to travel 300 percent further than spiders with their palps intact.

The post Weight of genitals reduces physical endurance in male orb web spiders, researchers find appeared first on Smithsonian Insider.

They observed that no plugs were ever formed during mating trials, but instead, females exposed to many males produced the amorphous plugs during the egg-laying process.

The post Female spiders produce mating plugs to prevent unwanted sex from males appeared first on Smithsonian Insider.

New high-speed videos of 14 species of tiny Mecysmaucheniid “trap-jaw” spiders have revealed that some species can snap their mouth parts shut at incredibly fast […]

The post Discovery: trap-jaw spiders snap prey with super-spidey speed and power appeared first on Smithsonian Insider.

In 1854, a curious-looking spider was found preserved in 50 million-year-old amber. With an elongated neck-like structure and long mouthparts that protruded from the “head” […]

The post These newly discovered pelican spiders will make you want to visit Madagascar appeared first on Smithsonian Insider.

Jonathan Coddington, Curator of Spiders at the Smithsonian National Museum of Natural History describes how photography has transformed the study of arachnids.

The post How photography has transformed the study of spiders and their webs appeared first on Smithsonian Insider.

Arachnophobes fearful of spiders jumping, creeping or falling into their beds now have something new to worry about. Some spiders might also glide in through the window. […]

The post Scientists study Skydiving spiders in South America appeared first on Smithsonian Insider.

At the Smithsonian’s National Museum of Natural History, curator of arachnids and myriapods Hannah Wood has examined and analyzed hundreds of pelican spiders both in the field […]

The post Pelican spiders: Ancient assassins that eat their own kind appeared first on Smithsonian Insider.

Different management practices using pesticides affect the diversity, number and ecological traits of ground spiders in apple orchards, a new study finds. Because spiders are viewed as good indicators of the quality of an entire ecosystem, the results reveal that organic orchard management may be better for local management and landscape characteristics when compared to those with pesticide use.

Two chemicals used as co-formulants in pesticides have been found to reduce the predatory behaviour of the wolf spider Pardosa agrestis, an insect predator found within agricultural landscapes. A third co-formulant was found not to affect the predatory behaviour of females and increased the prey behaviour of male spiders. This is the first time that pesticide additives have been shown to alter the predatory activity of a potential biological control agent of crop pests.

Certain female spiders are infamous for eating their partners after sex, but some actually pounce on suitors, fangs first, before mating.

With huge eyes in tiny bodies, baby jumping spiders have high-quality vision.

Soil is drying up all over the world, Google is dropping big bucks building a wind power transmission line, and a spider and centipede battle it out.

Spiders are great for pest control, except when they overtake your home. Here are some nontoxic solutions for dealing with them at your house.

Researchers have managed to pinpoint the time you're most likely to meet a spider.

New research finds mean spiders know what it takes to survive extreme weather.

A new study introduces 18 previously unknown spider species that look remarkably like pelicans.

A group of stick spiders in Hawaii evolves into the same three 'ecomorphs' every time it colonizes a new island or region.

Australian Reptile Park shares a creepy video of funnel web spiders emerging from an egg sac, just in time for Halloween.

Turret spiders, the minuscule relatives of tarantulas, pack a powerful punch for any insect venturing close to the castle walls.

When black widow spiders mate, the females often kill and eat the males. But the male spiders have found a way around this terrible fate.

Although nanorobots made of DNA have been developed before, these spiders can move over greater distances and follow complex commands.

Scientists still aren't entirely sure how the spiders make use of the carbon nanotubes, but their webs are the strongest ever recorded.

The EPA needs better pollution monitoring equipment, President Obama trashed hundreds of bikes, and someone designed a super box.

Pest-killing spiders behave differently after exposure to a common insecticide, a new study finds.

Spiders eat several hundred million tons of insects per year, a new study finds, a global feast rivaling the yearly meat intake of humans.

Many people don't see arachnids as 'cute,' but maybe they've never laid eyes on a peacock jumping spider.

And why a zoo is asking the public to help capture them.

For Anelosimus studiosus spiders, the storm survival checklist apparently includes a combative personality.

SPIDERS can invade your home during the summer as they escape the heat from outside. But when do they come indoors and how can you protect your home against spiders?

Hailing from all over Australia, the spiders were identified by a scientist who used to fear them

With the help of their webs, spiders are capable of foresight, planning, learning and other smarts that indicate they may possess consciousness

With the help of their webs, spiders are capable of foresight, planning, learning and other smarts that indicate they may possess consciousness

When heavy rains caused rivers to overflow and flood parts of Pakistan last year, it set into motion a surprisingly complex chain of events that scientists are only now beginning to understand -- and which may have

Although some may wish it so, a world without spiders would be a miserable place.

It took four years, one million spiders, and 80 people in Madagascar to make this cape, on view at London's V&A Museum -- and the color is natural.

The floods that submerged one-fifth of Pakistan's territory took more than six months to recede.

One of the most affected regions was the Sindh region, located in the western corner of South Asia. 

At their peak, the floodwaters were up to 20 feet deep. About 20 million people were displaced.

But apparently, people were not the only ones seeking shelter from the devastating floods. 

One of the unexpected side-effects of the flooding has been that millions of spiders climbed up into the trees to escape the rising floodwaters, turning them into futuristic-looking trees cocooned in spiders' webs.

The people living in that area claimed they have never seen this phenomenon before but were glad to discover that those cocooned trees were actually significantly reducing the numbers of mosquitos and thereby, the risk of malaria. 

It is thought that the mosquitoes were getting caught in the spiders' webs which would be one blessing for the people of the area, facing so many other hardships after the floods. 

Check out some of the stunning photos, released by the department of international development.