yes, it directly starts with a loss of 0 
its getting alive
and it moves
https://www.youtube.com/watch?v=opbjvah84ls
ok, the algo must be mader better, for continous action a3c seemed better, but first it needs to be setup correct. Help is welcome
the training of a real hardware instead of e.g. pong-game is a little bit different. Doing an action takes with real hardware some seconds and so getting enough samples to train on, means running the hardware for weeks, just doing mostly random things.Which then produces many,many gigabytes of data i cannot store anymore 
So i wrote a game, that uses the same sensor data, but i dont need real hardware. I hope i can move on faster now. here is the code, you can simply run with “python flobotics_game.py” on console and with tensorboard --logdir=/tmp/train you can watch in webbrowser the training results. Have fun and help is always welcome
EDIT: The game has now a TKinter graphical window to watch it train or play it
tensorflow is really cool, you can use it like lego. I got some working models which do its job after training only 1 hour on CPU (no GPU). Here some screenshots of 1 hour training and then the model does the job for 1 hour alone.
I post these pictures here, because there are not much out there, and i firstly thought my model would not work, because of the strange look of e.g. the loss. But it does All code is on github.
After working a while with these visual qr-tag detectors, i must say, that they dont really work well with ROS. But, after thinking a while, i got it, that it does not work with a neural network trained model. The reason is, the neural network needs the degrees of the phalanxes of one finger, even if there is no camera possible in this situation, or the tags are so big, that they disturb the movements.
My solution to this problem are 9-DOF sensor, MCPU-9150. The chip is only 4x4 mm in size and could be easily used on every phalanx. The smallest so far breakout-board i found is 16mm x 21mm x 1,5mm and i could print some adapters. Here a picture with the first and big adapters. 
Now i get the position of all phalanxes, even if its dark-night, where a camera couldnt see anymore And the neural network can work. I dont know of a finger or hand, that is able to detect its own positions of the phalanxes
Hi @flobotics,
Sorry I’m late to Discourse and to this discussion to suggest actually that.
At Centro Piaggio “E. PIaggio”, we used to use a IMU-based glove on top of the Pisa/IIT SoftHand, because there were no way to add encoders, since there are no rotating shafts between phalanxes in that hands. There is actually a publication about it, in case you want to read more:
Another releated and commercial project is Perception Neuron:
But it does
All code is on github.
On the other hand, I’m interested in the part where you used TensorFlow, how did you do the training, using the real hw?
Did you wrote somewhere all that process?
Thanks
hi @carlosjoserg , its never too late for some good info With tensorflow i use the deep-q net from deepmind, which played GO against Lee Sedol. I made it like a game. 
In the image (which is mostly black and old) there are six arrays displayed. I overlayed them in this picture to see it more clearly for humans. The white-strip shows the goal-angle and the white-box shows where it actually really is.
The next array shows the force or tension on the first wire-rope. Again, the white-strip shows the goal-tension and the white-box shows the actual tension. The same with the next array, which shows the tension of wire-rope number 2.
The tensorflow-ros code makes one-step, that means, its not continous.(A3C seemed better for that) But anyway, after that step it checks the actual state, which means angle, tension1 and tension2. It gets it rewards. Then it does the “normal” tensorflow way.
For better reproducibilty, i now use dynamixel servos, which got a load parameter instead of flexi-force sensors.
For an easy start, you can check https://github.com/flobotics/flobotics_tensorflow_game/tree/master/pixel_hunter_game/10x2 It does nearly the same “game”, but its faster (does not need a week to train, just an hour on cpu)
hope that was informal
SEARCHING INVESTOR for robotic humanoid hand. To build 5 fingers of different length (as a human has), getting 38 (58-2) servos to control it like, or even better, as a human and build 19 (54-1) smaller 9-DOF sensors (to connect to each phalanx). If someone is interested, please mail to info@flobotics-robotics.com. Thanks
I made a picture of a hardware-one-finger-setup and a gimp manipulated picture of hardware-five-finger-setup to show that hardware with 38 servos is not so big as someone thinks. They would easily fit into a small box, and the cabling could go around curves, so that the box could be mounted everywhere. 38 servos, because the thumb has one phalanx less than the other fingers.
Click on the pictures to see it in full size.
Original picture of hardware-one-finger-setup
Manipulated hardware-five-finger-setup:
The advantage of 38 servos is, you can move all fingers in every position a human can do and beyond. That means, you can move every phalanx e.g… 90 degrees to the opposite side of the hand, try that with your hand
Controlling of all 38 servos is then done with artificial intelligence, like described here http://arxiv.org/pdf/1603.06348.pdf or here http://arxiv.org/abs/1603.02199
Created a 3d-printed base for all five fingers and put the wire-ropes into 1.8mm small tube. The base should be much smaller and made with aluminium and with screw threads, so one can easily fix all the fingers to the base with a small screw. Or perhaps even better, the base made out of a harder silicon, so that its more like human.
You can see that the fingers have all the same length and so the hand is not really like a human hand, but can do all the moves.
Here i have put the fingers and the base into a silicon hand.
Now i need 38 servos 
That is how it should look like. O.K. with a nicer box and 38 servos in it, it would look much nicer, but to get a image of it.
You need to click on the picture to see it full
i made a new and smaller hand-base with a hole to fix a small pipe into. As you can see, i used the double joint from the flobotics finger as a wrist-joint, which would add 4 more servos to move the wrist-joint, but its possible.
I also cut the thumbs metacarpal “bone” and removed one phalanx . 
It all fits into the silicon hand, only the tube-hole, which could be made smaller if everything would be aluminium.
i made it out of aluminium with grub-screws to easily fix the fingers. I am not a cnc expert, so professionally it would be nicer and perhaps even smaller.
For the thumb i needed to add the removed phalanx or it had not the freedom like a normal hand, e.g. it couldnt touch the other fingers tip. I cut the last phalanx, because the thumb has one less then other fingers, but the count of joints are the same as a normal finger.
The next part would be the box with the servos inside.
The proto-prototype of the servo-box is better than i thought. As always, with some cnc-made aluminium parts, the box could shrink in its size to around 40 cm in height and 30 cm in width. Then build into a nice electrical cabinet. On the picture you can see, that it has 5 steps, on every step there will be 8 servos for one finger. Then you can use it on e.g. warehouse vehicles and grab stuff.
I got a hardware setup to power 40 servos and only use one USB2Dynamixel adapter. So i can address all 40 servos directly with one control software, only need to handle one usb-port.
Here is a picture, where you can see that every 6-port-hub is connected with its own power supply and every 6-port-hub is then inter-connected with each other only with the data-line of the 3-pin connector.
On every 6-port-hub then are 4-hubs available for servo-use.
Wow, thats how i thought it should be. A little nicer optics would be better, but it works. I put everything together inside a little box and put the robotic humanoid hand on top of it. The wire cables can be moved very easily. All 8 servos are recognized through one usb-port.
Pinging motor IDs 1 through 25…
Found 8 motors - 8 AX-12 [1, 2, 3, 4, 5, 6, 7, 8], initialization complete.
Nice things the finger can learn with ai . Learn to move the finger to a numpad number and press it, response/reward comes from keyboard itself. Or i build a little box with buttons, switches and so on, connect it to a arduion/rpi and get respons/reward for a convnet. Tipps and help is always welcome.
Back to the roots. @samiam The AX12-A servo does only send load value while it is moving, when you stop it, it sends a zero value. Thats not very good for usage. So i use again the flexiforce sensors to get the wire cable tension of every single wire cable. So a ai model can use this information for howto move the hand and also could “feel” if someone/something touches a finger.
Here a picture of the new setup with flexiforce sensors, the box is not getting bigger, which is positive, but more hardware is used.
1 Like
Here i move the finger only per manual control, thats why its so slow and not in one-move. You can move it as fast as the servo is. Here you can see how it moves Like i said, a little deep learning would be good, if someone knows a implementable code, please let me know
Tipps for deep learning would be nice. I can tell, that i got 8 force values, and 8 values (negative and positve) for 8 servos to control speed. Also i could use a video-stream from the finger as input.
As output the best would be a regression-output of 8 different values for each servo. But nearly every neural net is a classification net, and then i would need nearly endless outputs for every speed and every servo, also it could be that more than one servo is moving at the same time, so i would need even more classification outputs ?
To put all this information into a picture, i thought about appending the info to the video-stream of the finger. I could add two other “info-pictures” on top of the finger-video-stream with the information about what the finger should do (press button 1, press button 2, or release finger from button). This i would do with a image of a number.
The next information is the force value (or wire tension value) from the force-sensors and a want-to-have force value from the user. So i simply would use 16 lines of 1pixel height and 1024 pixel in length (because there are 1024 force values). The first pixel line is the force i want-to-have on sensor1 and pixel line 2 is the actual force value from the sensor.
This video-stream i could then feed into a neural network.
So the neural net could get its rewards for matching force values and for the signal that comes from the keyboard if a button is pressed. Punishments if the force values are less then the want-to-have force value.
Here are some pictures that shows it. The numbers are what button it should press and X means it should release the button. The black pixels showing the force values.
I wrote a ROS node which captures the webcam stream with opencv and then i added two “images” above the webcam image.The first “image” is 50x50 pixels and shows the command. The second “image” is 32x512 pixels. I choose this, because the webcam video is 640x480 pixels, and so i can simply add 16 arrays of 1024 pixels. Then the webcam image is the third image. This image stream i then publish and can simply see it in rviz.
You can see the 16 black pixels above the webcam image and under the command-image. But as you can see, they are only pixels and not good to see, so my question is now, would a convnet be able to “see” and keep apart these pixels, or are they too small?