Erasure codes are one of those seemingly magical mathematical creations that with the developments described in the paper XORing Elephants: Novel Erasure Codes for Big Data, are set to replace triple replication as the data storage protection mechanism of choice.
The result says Robin Harris (StorageMojo) in an excellent article, Facebook’s advanced erasure codes: "WebCos will be able to store massive amounts of data more efficiently than ever before. Bad news: so will anyone else."
Robin says with cheap disks triple replication made sense and was economical. With ever bigger BigData the overhead has become costly. But erasure codes have always suffered from unacceptably long time to repair times. This paper describes new Locally Repairable Codes (LRCs) that are efficiently repairable in disk I/O and bandwidth requirements:
These systems are now designed to survive the loss of up to four storage elements – disks, servers, nodes or even entire data centers – without losing any data. What is even more remarkable is that, as this paper demonstrates, these codes achieve this reliability with a capacity overhead of only 60%.
They examined a large Facebook analytics Hadoop cluster of 3000 nodes with about 45 PB of raw capacity. On average about 22 nodes a day fail, but some days failures could spike to more than 100.
LRC test results found several key results.
- Disk I/O and network traffic were reduced by half compared to RS codes.
- The LRC required 14% more storage than RS, information theoretically optimal for the obtained locality.
- Repairs times were much lower thanks to the local repair codes.
- Much greater reliability thanks to fast repairs.
- Reduced network traffic makes them suitable for geographic distribution.
- LRC test results found several key results.
- Disk I/O and network traffic were reduced by half compared to RS codes.
I wonder if we'll see a change in NoSQL database systems as well?
Related Articles
- Erasure Coding vs. Replication: A Quantitative Comparison
- Ceph - a distributed object store.
vinces99 writes "Small electrodes placed on or inside the brain allow patients to interact with computers or control robotic limbs simply by thinking about how to execute those actions. This technology could improve communication and daily life for a person who is paralyzed or has lost the ability to speak from a stroke or neurodegenerative disease. Now researchers have demonstrated that when humans use this brain-computer interface, the brain behaves much like it does when completing simple motor skills such as kicking a ball, typing or waving a hand (abstract). That means learning to control a robotic arm or a prosthetic limb could become second nature for people who are paralyzed."
Read more of this story at Slashdot.
It's not quite a quantum internet — yet. But researchers at Los Alamos National Laboratory in New Mexico have developed a new, ultra-secure computer network that is capable of transmitting data that has been encrypted by quantum physics, including video files. The network, which currently consists of a main server and three client machines, has been running continuously in Los Alamos for the past two and a half years, the researchers reported in a paper released earlier this month. During that time, they have also successfully tested sending critical information used by power companies on the status of the electrical grid. Eventually they hope to use it to test offline quantum communication capabilities on smartphones and tablets.
- Computer network
- consumer devices
- Cryptography
- cryptography
- Emerging technologies
- encryption
- Formal sciences
- Los Alamos National Laboratory
- Mathematical sciences
- New Mexico
- Nokia
- Olympus
- Physics
- quantum communications network
- Quantum cryptography
- Quantum information science
- quantum internet
- Quantum key distribution
- Quantum mechanics
- Quantum network
- quantum network
- quantum test networks
- real-time communications
- Richard Hughes
- secure communications
- smartphones
- Technology
- ultra-secure computer network
The Diamond Bionic Eye
Google Tech Talk February 1, 2013 (More info below) Presented by Professor Steven Prawer, Director, Melbourne Materials Institute, The University of Melbourn...
From:
GoogleTechTalks
Views:
2118
45
ratings
Time:
01:00:09
More in
Science & Technology
- Artificial organs
- Australia
- Biology
- Biomedical engineering
- Blindness
- Emerging technologies
- Eye
- Health
- http://gdata.youtube.com/schemas/2007#video
- Macular degeneration
- Medicine
- Melbourne Materials Institute
- Neuroprosthetics
- Retina
- Retinitis pigmentosa
- Steven Prawer
- University of Melbourne
- Visual prosthesis
Last week we heard about the Argus II, a device that can restore partial sight to some blind people, and this week a new retinal prosthesis is promising to go one step further. While the Argus II relies on glasses, an externally-mounted video camera, and a separate processing box, the Alpha IMS system detects light coming into the eye via electrodes implanted underneath the patient's retina, before feeding it into a microchip that sends the signals to the brain. The brain then processes the data as it would organic signals from a healthy eye, and the patient sees a black and white image. There's also a dial fitted behind the ear for adjusting brightness, and the whole system is powered wirelessly by a pocket battery.
Developed by...
- Alpha
- Anatomy
- Artificial organs
- Biology
- Dell
- Emerging technologies
- Germany
- Health
- HTC
- Hungary
- Implants
- Medicine
- natural processing power
- Neuroprosthetics
- Prosthesis
- Prosthetics
- Retina
- separate processing box
- United Kingdom
- University of Tübingen
- vision-processing parts
- Visual prosthesis
- Visual system
adeelarshad82 writes "Oculus VR Rift is a one of the seventeen kickstarter projects to raise more than a million dollars in 2012 and a recently published hands-on shows exactly why it was so successful. Using Oculus VR Rift with the upcoming Infinity Blade and a modified version of Unreal Tournament 3, the analyst found that the 3D effect and head tracking provided a great sense of immersion. At one point while playing Infinity Blade, the analyst describes walking around the guards and watching their swords shift as he stepped, seeming like they were inches from cutting him. While he felt that the demo was impressive, he found that the software limitations made the whole experience a bit unrealistic. Needless to say that Oculus Rift is a long way from hitting stores but Oculus VR is getting ready to ship developer kits."
Read more of this story at Slashdot.
- ad network
- Amazon
- analytics tools
- Apache Hadoop
- API
- Big data
- big data applications
- BitDeli
- business intelligence
- Cloud Architect Todd Papaioannou
- Cloud computing
- Cloud infrastructure
- company claims developers
- Computing
- Concurrent computing
- data-processing
- developer tools
- Emerging technologies
- Europe
- Facebook HBase
- Hadoop
- HBase
- Java
- Java
- Jeff Carr
- Jonathan Gray
- Keen IO
- machine learning
- MapR
- mp3
- Newtown
- Precog
- Python
- Ruby
- Software
- SpringSource
- StatsMix
- Structured storage
- Technology
- Ville Tuulos
- Web applications
- web services
- Yahoo!
Mind-controlled Machines: Jose del R. Millan at TEDxZurich
The idea of controlling machines not by manual operation, but by mere "thinking" (ie, the brain activity of human subjects) has always fascinated humankind. A brain-machine interface (BMI) makes this truly possible as it monitors the user's brain activity and translates their intentions into actions, such as moving a wheelchair or selecting a letter from a virtual keyboard. The central tenet of a BMI is the capability to distinguish different patterns of brain activity each being associated to a particular intention or mental task. This is a real challenge which is far from being solved! BMI holds a high, perhaps bold, promise: human augmentation through the acquisition of new brain capabilities that will allow us to communicate and interact with our environment directly by "thinking". This is particularly relevant for physically-disabled people but is not limited to them. Yet, how is it possible to fulfill this dream using a "noisy channel" like brain signals? Which are the principles that allow people operate complex brain-controlled robots over long periods of time? Jose del R. Millan is the Defitech Professor at the Ecole Polytechnique Federale de Lausanne (EPFL) where he explores the use of brain signals for multimodal interaction and, in particular, the development of non-invasive brain-controlled robots and neuroprostheses. In this multidisciplinary research effort, Dr. Millán is bringing together his pioneering work on the two fields of brain-machine interfaces <b>...</b>
From:
TEDxTalks
Views:
36
3
ratings
Time:
14:15
More in
Science & Technology
- Barcelona
- Brain
- Brain–computer interface
- Ecole Polytechnique Federale de Lausanne
- Emerging technologies
- European Commission
- Health
- http://gdata.youtube.com/schemas/2007#video
- Human–computer interaction
- Idiap Research Institute
- Idiap Research Institute
- International Computer Science Institute
- Italy
- Joint Research Centre
- media coverage
- Millán
- Neuroprosthetics
- Neuroscience
- R. Millan
- Robot
- Scientific American
- Spain
- Stanford
- Switzerland
- Technology
- TED
- Time
- United States
- Virtual reality
If the thought of a robot apocalypse is keeping you up at night, you can relax. Scientists at Cambridge University are studying the potential problem. From the article: "A center for 'terminator studies,' where leading academics will study the threat that robots pose to humanity, is set to open at Cambridge University. Its purpose will be to study the four greatest threats to the human species - artificial intelligence, climate change, nuclear war and rogue biotechnology."
Read more of this story at Slashdot.
- ai
- artificial intelligence
- Artificial intelligence
- Artificial intelligence in fiction
- Cambridge University
- Computational neuroscience
- Department of Defense
- Emerging technologies
- fancy schmancy software
- Knowledge
- Open problems
- Pentagon
- Philosophy of artificial intelligence
- Raytheon
- Robot
- Science
- Technology
- teh internet
- Terminator Center
- U.S. military
- United States
- US Federal Reserve
Perhaps it is the onset of delirium caused by trying to properly report a 30,000 attendee conference while also (and no less properly) reporting the colorful after-hours of New Orleans, but after a few days at SfN 2012 I have acquired the impression that this huge mass of brain scientists, when focused and sober, is capable of all sorts of wonder on which an apprentice science fiction author would feast. None of the press so far seem to harbor ambitions for literature, but if you had to bet a grant on who secretly does, bet on the absent freelancer – we will leave him unnamed – who carries around the convention center a fresh mint julep and feeds the mint leaves to the mouse saved from a laboratory that rides sentry on his shoulder.
There are plenty of hot topics to choose from, but his first book would probably be about neural optogenetics. A combination of optical and genetic research methods, optogenetics involves shooting lasers into particular brain tissue to inhibit or disinhibit its operative cells. Since its breakthrough about two years ago, the method has advanced to the point where researchers now talk about perfecting it and applying it. It’s fascinating tech, but does it amount to mind control, as some YouTube commenters might have you think? Not exactly.
- Biology
- Blink
- Bradley Voytek
- Daniel McDonnall
- electronics
- Emerging technologies
- Eye
- FDA
- Federal Drug Administration
- Health
- Knowledge
- lasers
- Lawrence DeGeest
- National Eye Institute
- Neuroprosthetics
- Neuroscience
- Neuroscience
- Neurotechnology
- New Orleans
- Optogenetics
- Ripple
- Science
- SfN 2012 I
- shooting lasers
- Society for Neuroscience Conference for Motherboard
- Society of Neuroscience
- Timothy Verstynen
- YouTube