View source for P5.js as an Art Medium

''This is an artist page by [[Kristine Fernandez]]. This project was part of the 2023 [[CATS+ Computer Club]] cohort. 

My intent during the [[CATS+ Computer Club|CATS+ residency]] was to provide coding support for the residents to support their work. At the end of the day, p5.js was at the root of all of the output that I had a hand in.  

== p5.js? ==
[https://p5js.org/ p5 is a javascript library] used for creative coding, created to make programming art accessible to more people whether you have a coding background or not. I’m drawn to p5 in particular because while the different methods are relatively simple to understand, the output can be really complex and interesting. I like that p5 can feel more like a natural language extension where I can focus on what I’m creating instead of getting bogged down by some of the walls that typical programming can run into (fighting libraries, compiling issues, weird gotchas).    

I had read a bit about flow state (e.g. related to the Japanese concept of [https://medium.com/@jp/how-to-achieve-your-flow-state-2265ca948ac3 ikigai] or Mihaly Csikszentmihalyi’s work on flow in relation to happiness) and coding is one of the mechanisms where I’m able to feel that blissful feeling of reaching a flow state. Developing in p5 in particular, allows me to find that flow state more frequently because of its ease of use.   

[https://kristinefernandez.com/blog/c-3-2-2-2-3 Some of the work that I’ve done in my early p5 experimentation mimics this flow state] that I feel in the act of coding, typically creating visuals that are pleasant and often tied to sine and cosine functions which give the visuals a wavelike repetition.   

[https://fernakr.github.io/feelings/life/ Subsequent work has been to use p5 more as a tool to make a concept a reality] and that is how I approached the work within this residency.    

== In Action ==
At its base, p5 is a drawing library (e.g. draw a square, rotate it, draw a cube and move it based on user input); however, because of its robust tooling, there are many other ways that p5 can be used for an interactive or dynamic piece that ended up being well-suited for everything we needed.

=== Stop Motion Command Center / Progress Bar ===
[[kit-bashing // input cycles|Sara Roma]] needed a panel that the audience could interact with to affect her stop motion piece. During one of our cohort discussions, I mentioned that progress bars could be programmed at the discretion of the programmer who built it so they can give us a flawed perception of progress. From this thought, [[kit-bashing // input cycles|Sara]] conceptualized the idea of a progress bar with a mind of its own so the audience might think that they are starting to understand what impacts the progress percentage but actually it jumps around somewhat arbitrarily. 

https://fernakr.github.io/moha-cats/stop-motion/?duration=0.25 (To start, tap the spacebar.)

==== Progress ====
The progress bar is the primary p5 portion of this piece. It was surprisingly difficult to create the seemingly random increasing and decreasing of progress. I created some randomized inputs that determined whether the progress bar would increase or decrease; but I also wanted the duration parameter to control how many minutes it took the progress bar to reach completion, which resulted in a mathematical quandary where the probability of increasing, decreasing or staying the same was based off whether the progress bar was greater than or less than where it should be based off the intended duration.

==== Commands ====
See [[kit-bashing // input cycles|Sara Roma’s wiki]] for more information about how the commands affected their performance. My main intent here was to create a visually engaging piece that invited user interaction while also giving Sara enough context to how long each action had remaining.

==== Glitch Disconnect ====
When the progress bar reached 100%, I really wanted to create a visual that surprised the audience so I came up with this idea of the screen glitching out and “disconnecting” to really sell this idea of the panel being functional outside our normal idea of what a web browser is capable of. To do this I used a [https://www.npmjs.com/package/html-to-image javascript library] to take a screenshot of its current state, inserted that screenshot into the p5 canvas, then used p5 <vertex shaders to distort that image output.  

=== Video ===
The source is a compressed version of the dance video from a workshop led by Itai Amor. I ran the video through p5 and experimented with shader effects and an AI-generated soundtrack. 

https://fernakr.github.io/moha-cats/video/ (To cycle through effects, click the video.)

==== Shader Effects ====
Our robotic movements in the videos and the video compression created some really interesting visual artifacts that I thought would be really interesting to run through some shaders, especially ones that could get affected with realtime user input like mouse movement or just the flow of time.
==== Sound ====
AI is a very prominent topic in the art world right now, and I won’t get too deep into it, but I did see a website called [https://melobytes.com/ Melobytes] that uses AI to create all sorts of outputs based off all sorts of inputs so I thought why not see what happens when you run the video through the AI. The “music” in the background is the output of running the compressed video through its video-to-song service. I did have to lower its tolerance quite a bit so I’m not sure if it really had an extremely close tie to the visual, but it did produce some eerie sounds.

== The Archive == 
During the course of the residency, we knew we wanted to document our experience so we collected all our notes (both physical and digital) and put them on display during the showcase. [[Thermochromic Ink & Soft Interactivity|Serena]] had the idea to create a gradient wall that incorporated the physical-to-digital manifestations of our work together. I knew I could do the digital portion in p5 so that became my task.  

<b>Giphy:</b> https://fernakr.github.io/moha-cats/archive-giphy 

This was the original MVP for creating the general look and feel of the archive wall. I knew there was going to be video as an option, so I figured Giphy API would be a good first pass at testing the concept. 

<b>Cats+:</b> https://fernakr.github.io/moha-cats/archive/ 

Once we collected all of our digital artifacts, I updated the visual a bit to give the artifacts a bit more depth and layering for more visual interest and also incorporated a little easter egg homage to [https://moha.wiki/w/images/c/c5/MoHA_CATS2_2023_Fernandez_berrychair_0001.png Berry Chair] who was an honorary resident (s/o to Kellyn for that idea!).

== Appendix ==
=== Shaders ===
Shaders are code that is run on the GPU and used for things like post-processing filters e.g. they can be used to grayscale or distort your image or video. p5 uses OpenGL Shading Language (GLSL) but there are shaders in other platforms like visual effects or in javascript libraries like [https://threejs.org/ ThreeJS].    

While p5 is code that draws onto the canvas, shader code is run on every pixel of the canvas. So a vertex shader is a shader that affects the position of each pixel. This is good for distortion effects [https://aferriss.github.io/p5jsShaderExamples/4_image-effects/4-19_slitscan/ like this slitscan example]. Fragment shaders on the other hand affects the color of each pixel. [https://aferriss.github.io/p5jsShaderExamples/4_image-effects/4-12_threshold/ So this is good for any color-based effects like this threshold example].

*[https://itp-xstory.github.io/p5js-shaders/#/./docs/what-are-shaders Read more about shaders]
*[https://github.com/aferriss/p5jsShaderExamples See more shader examples]
*[https://www.shadertoy.com/ See some absolutely insane shaders that defy my own understanding of shaders]

== Related links ==
*[https://kristinefernandez.com Kristine's website]

{{Styled category|CATS+}}
[[Category:CATS+ Computer Club]]
[[Category:Article]]