Towards a Bird-Computer Interface!

[NightEule5]

That's a smart dispensing mechanism. Nice work!

As far as Arduino, the board would be rather cumbersome and expensive for what you are trying to do, yes. I guess what I was saying is you could use the Arduino language (or C/C++, since the Arduino language is kind of slow) to program rather than using Assembly. But, I guess if you are confident with assembly it probably wouldn't be an issue. I just have never got around to learning that yet. I'd imagine using Assembly directly would be much more efficient.

As a side note, I'm not familiar with the micro you're using, I guess I'll have to look it up.

 

[NightEule5]

Oh, nevermind. It's not even AVR. I guess Arduino wouldn't work regardless afaik.

 

[Bryce]

Thanks!

Understandable to not immediately realize it wasn't an AVR, since it is an Atmel (AT...) chip; I don't think Atmel / Microchip themselves really advertise that they make a line of 8051s all that well. I didn't know either until I went looking for the cheapest 8051s I could find (knowing that the needs of this project were pretty minimal.)

I actually did think about using C, but had some problems with SDCC and kinda wanted the efficiency of assembly anyway for this program.

 

[Kiwibird]

This is so cool to watch the process of development on!

I feel like a knuckle dragging cavewoman by comparison. About the best foraging toy I've come up with is drilling a hole in a cup+saucer and hanging them on a chain together lol (best we stick to the store bought puzzle toys/trick props for a real challenge). Ruby is at a huge advantage to the other parrots on this forum being taught more complex problem solving skills. Do you plan to integrate shapes/colors into the computer interface eventually? Like correctly identify a shape or color, get a treat? A touchscreen interface perhaps using an old tablet running on Linux? I think that's the OS my husband uses on the computers he builds that is very "customizable" compared to Android or IOS (I have no idea what the proper "terminology" is, but I hope you get the idea I'm trying to convey).

 

[Bryce]

This is so cool to watch the process of development on!

Thanks!

Ruby is at a huge advantage to the other parrots on this forum being taught more complex problem solving skills.

We'll see. She's trying out the prototype today. I've seen her press the button several times, but she doesn't seem to have made the connection yet between pressing the button and the presence of a treat in the chute. She does know the sound it makes because she will come and investigate the chute after hearing the sound when I press the button, but she hadn't gotten it all figured out yet when I left for work. I don't know what expression a parrot gets when its mind is blown, but I assume sometime today or tomorrow she'll be making it when she figures out that pressing a button can make food appear where there was no food before. Magic! A whole new paradigm of Making Food Appear, distinct from the previously known sciences of Banging On Things And Hoping Seeds Fall Out and Destroying Stuff With Your Beak 'Cuz Maybe There's Seeds Inside.

Do you plan to integrate shapes/colors into the computer interface eventually? Like correctly identify a shape or color, get a treat?

Yes. I'll need to add some more buttons, though. Right now there is only one button and it directly triggers the dispenser. The dispenser can also be activated by a computer, though, so this sort of thing is definitely possible (and in the plan.) I'm also planning on teaching her to play "Pong" with a progressive system of games, tentatively:

Stage 1: Press a button to get food (here now)
Stage 2: Press a button to get food when a visual cue (the pong game) is shown on a monitor.
Stage 3: Press one of two buttons to get food, depending on the visual cue shown.
Stage 4: Move a paddle to hit a ball, get food.
Stage 5: Do this multiple times in a row to get food.
Stage 6-onward: Add a computer opponent of increasing difficulty, bouncing the ball back.

A touchscreen interface perhaps using an old tablet running on Linux?

I don't think a modern capacitative touch screen would work all that well with a beak, but I could be wrong. Currently I'm using arcade machine buttons, figuring that if they stand up to frustrated kids banging on them, they might survive The Beak.

There would definitely be some advantages to touch screens, if they could be rugged enough and sensitive to beaks and feathered heads. Some of the older-style touchscreens using IRED/Photodiode arrays, acoustic triangulation, or a resistive sensing element might be better suited. I think they still make them for kiosks and ATMs, but I'm not sure how expensive they are. The arcade machine buttons I got were really cheap, but they also were not very high quality (over half of them were defective) so I'm definitely thinking about other options. Oddly, eBay didn't have any joysticks specifically designed for beaks.

I 100% agree that Linux would be appropriate. I'm thinking at this point of running it on a BeagleBone, a cheap single-board computer, but if it turns out that modern touch screens can survive beaks (and be triggered reliably by them) a tablet would probably be a good option.

 

[NightEule5]

I don't think a modern capacitative touch screen would work all that well with a beak, but I could be wrong. Currently I'm using arcade machine buttons, figuring that if they stand up to frustrated kids banging on them, they might survive The Beak.

Ya, I don't think so. The reason (you probably know this already, lol) they work with fingers is because of the fact that our skin conducts electricity. I don't think a break would, but I'm not sure. It would probably be comparable to trying to type on your iPad with your fingernail (I just tried this to confirm, lol).

I was thinking resistive touch might work, but you already said that.

If you do this, you may want to put a screen protector on the screen, as I think a parrot beak would scratch it pretty fast. But thinking about it, they might be able to rip that off, so Idk.

 

[Bryce]

Well, if I had to design a parrot touch screen, I suppose I'd go with something that can have a tempered glass front, to resist scratching and biting. But it has to work with a non-conductive "stylus" (i.e. beak) as you mention. I imagine a peck could be localized acoustically using microphones placed at the corners of a piece of glass (basically, if the top microphone hears the peck/tap before the bottom one, it must be nearer to the top; if the left one hears the peck before the right one, it must be nearer the left of the screen). This might be quite a bit cheaper than the other kind of highly-durable touch screen I know about, which uses two arrays of roughly collimated IR LEDs and phototransistors to detect an object when it breaks the grid of beams, and maybe also higher resolution (though for this kind of thing, I doubt resolution is that important.)

 

[NightEule5]

I actually did think about using C, but had some problems with SDCC and kinda wanted the efficiency of assembly anyway for this program.

Well, better you than me. I have almost never (maybe once) tried to learn assembly, but if you know it that's great. The most I've gotten in that direction C++ and a little bit of C. I really like C# (when I'm doing Windows apps).

 

[NightEule5]

Well, if I had to design a parrot touch screen, I suppose I'd go with something that can have a tempered glass front, to resist scratching and biting. But it has to work with a non-conductive "stylus" (i.e. beak) as you mention. I imagine a peck could be localized acoustically using microphones placed at the corners of a piece of glass (basically, if the top microphone hears the peck/tap before the bottom one, it must be nearer to the top; if the left one hears the peck before the right one, it must be nearer the left of the screen). This might be quite a bit cheaper than the other kind of highly-durable touch screen I know about, which uses two arrays of roughly collimated IR LEDs and phototransistors to detect an object when it breaks the grid of beams, and maybe also higher resolution (though for this kind of thing, I doubt resolution is that important.)

Tempered glass would be good, but is there screen with that built in that you can get? Can you even buy Gorilla glass (is that still a thing?) if you're not a big corporation?

I don't think I've heard much of those other types of screens, but those could also work.

 

[Bryce]

Well, better you than me. I have almost never (maybe once) tried to learn assembly, but if you know it that's great. The most I've gotten in that direction C++ and a little bit of C. I really like C# (when I'm doing Windows apps).

Understandable. Using assembly language here is saving maybe $1.00 in materials cost. It's probably not actually justified. C would be more appropriate - maybe I should have spent more than fifteen minutes trying to get SDCC to work, haha. However, it was an interesting experience to do it the old-fashioned way. This is the first time I've written assembly language since college, and that was for a different processor, so I certainly sunk more time into this than if I'd gone with C.

For small microcontrollers, I don't think C++ offers much in the way of advantages over C. The big advantage of Arduino IMHO is the libraries and the base of peripheral support. Virtually everything you can buy from Sparkfun or Adafruit (or have shipped from Shenzhen for a couple bucks), someone has already written open source Arduino libraries to control. That can be really handy if you want to, say, interface with WiFi.

It's cool that you like C#. I don't use it much though myself (I program Python for computer stuff, both for my work automation needs and for hobby/household stuff at home). I used C# for a project I did in a research rotation and I don't recall it giving me any problems. I liked it better than Java at least - it kinda seemed like a better Java than Java. There are open source implementations of C#, so if you really wanted to implement parrot games in it, you could presumably use a Raspberry Pi, Beagleboard or other Linux-capable single-board computer. (Or a Windows-based Single Board Computer, which they apparently have now; someone in our lab got one recently.)

Tempered glass would be good, but is there screen with that built in that you can get?

There might be, but I was just imagining getting a sheet of the glass and putting it front of any old screen, with a small (~1mm) air gap between them. If the glass is thin and the parrot only has to peck at centimeter-sized objects, I don't think that would present any problems.

Interestingly, the proliferation of phone repair as a sort of cottage industry means that the supplies and tools for laminating glass, touch panels, LCD etc together have become quite available. I don't know if they're gorilla glass, but you can buy small tempered glass plates for phones and tablets for a few dollars. Also, I don't have one, but I've seen machines for doing vacuum lamination getting into the quite affordable range, so if it were necessary to make some kind of custom assembly, it would be feasible (for a company making kits or the finished product, I mean, not an individual only building one - "quite affordable" here being hundreds of dollars, which is why I don't already have one just in case )

 

[Bryce]

Ruby figured it out:
[ame="https://www.youtube.com/watch?v=MOrhdxujU70"]African Grey Parrot Presses a Button to get Treats - YouTube[/ame]

 

[Bryce]

I wrote the computer-side software for this, so that a computer can control the dispenser. It actually can address more than one dispenser on the same serial interface, so you could have say one for pistachios, one for sunflower seeds, one for walnut pieces, etc if you wanted to keep things varied.

So, it's likely I'll be working on making the next stage of the game this weekend! Exciting.

 

[Anansi]

Wow! Most impressive on both sides! I just read through this thread. On the technology front, there are times you might as well have been speaking another language. Hahaha! But fascinating stuff. On the bird learning and association side of things, I felt far more conversant and have been looking forward to the outcome of your project development.

Loved seeing the video of Ruby figuring out the button=treat scenario. I'll continue watching your and Ruby's progress with great interest.

 

[Bryce]

So, I've been pretty busy last week (and weekend), and didn't get to make as much progress on this as I'd hoped, but I did start working on the software. I now have a Python library on the computer with which I can control the treat dispenser. Also, I want to make a pong-like game, and I realized that a pair of arcade machine buttons aren't really the most ergonomic thing for that, especially if you have to press them with your head. So I'm looking into lever mechanisms that are optimized for beaks.

I've got a working prototype of a five-position switch (hard right, right, center, left, hard left) made with computer mouse buttons, which are very cheap and rated for a lot of cycles (imagine how much clicking you do with a computer mouse!), but I'm not sure if I'll stick with this design because it's kinda fiddly to assemble it. However, the action (pulling/moving the lever back and forth) seems much more natural than pressing a button. Certainly Ruby did figure out how to press a button, but I want to make the game intuitive and accessible to smaller psittacines who might have more trouble pressing a button.

 

[Bryce]

I made up a small batch of the new surface-mount boards, replacing the previous mess of electronics for the feeder. Still swamped at work, but I have some time off coming up in which I plan to make more progress on this.

 

[NightEule5]

Ooh! Very nice looking!

 

[Bryce]

It turned out that the connectors I had planned to use (GX-12 6-pin "Aviation" style connectors) are made of zinc alloy. They're plated, but I wouldn't want to chance a bird getting ahold of one and eventually gnawing through the plating. So, I've decided to switch to a "Mini DIN" connector, similar to the ones used for S-Video and PS/2 keyboards/mice (though not the exact same connector as the PS/2, for fear of causing confusion.) I'll have new circuit boards made to accommodate the new connectors, though of course that's going to push back the prototype delivery schedule. Sorry for the inconvenience, but I'm sure this is the right call.

 

[Anansi]

Definitely the right call. I'm quite impressed with all of the hard work and diligence that you are putting into this. Both in terms of the ease of interface and, even more importantly, the safety of the birds.

 

[Bryce]

Not fancy to look at, but I coded a simple "Pong" like tennis game. It has progressive difficulty levels, and adjustable incentive/reward schedule.