2Artist

The common conception of what an ‘artist’ does involves the production of creative work. This very broad definition encompasses other, narrower labels supported by social conventions and organisational niches, including fine artists, medium-specific artists, contemporary artists, artists who work for hire (e.g. designers, illustrators, and creative directors), and artists as defined by their industries (e.g. film directors and fashion designers). Artists who work with advanced technologies cut across all of these domains, frequently operating across several simultaneously. Currently, artistic work is organised depending on the economic specificities and societal expectations attached to each specific genre or form. The entanglement between how artists work and what they produce enacts a complex dynamic between the integration of AI into individual and organisational workflows, setting up a dynamic of feedback and emergence that exceeds the framing of automation and the question of the replacement of artists by machines. This chapter refers to artists working across the AI tech stack, engaging with the application and model layers through consumer-facing and specialist tools.1

In order to do justice to the breadth of unfolding transitions and scenarios in the context of AI, a comprehensive treatment of the different operational planes that constitute the structural core of AxAT artistic work is necessary. FAE identifies labour, crafting, aesthetics, systems, worldbuilding, and tech development as the six planes according to which the unique qualities of artistic work can be meaningfully rendered in relation to AI technology as a medium. The question of what happens on these planes when AI systems are deployed may offer insights into wider questions AI technologies are posing for society.

A framework for public claims on resources

The increasing integration of AI across various sectors is sparking widespread concern over the fear of becoming obsolete (FOBO). According to an IMF report, 60% of workers will be impacted by AI through job automation, and there will be changes in how critical decision-making such as hiring and disciplinary actions are operated.2 Many artists are nervous about their practice and skills becoming irrelevant as a result of generative AI outputs: DACS’ report on AI and Artists’ Work reported 77% of artists felt that AI will replace jobs and opportunities.3 The increased emphasis on labour rights is underscored by recent high profile collective bargaining battles such as the 2023 Writers' Guild strike in Hollywood.

Concurrently, government efforts, such as the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence in the US and the EU AI Act are attempts to use regulatory frameworks to safeguard workers whilst legislating other guardrails for AI applications.4 Whereas artistic labour that exists within commercial and formal industry sectors could fall under these provisions (because it is comparable to other forms of wage labour), what sets a good deal of artistic labour apart is the fact that it exists in regulatory no-man’s land, and, as such, represents the ‘worst case’ scenario of regulatory efforts falling flat.5 One regulatory regime that safeguards artists is IP rights, expressed as a means of recognition and value of artistic labour.

All media is now training data.

- Herndon Dryhurst Studio

The AI models that we have today derive from the human labour which comprises its training data. Large models are trained on vast quantities of text and imagery originating from the long arc of human cultural production, from chat rooms to digitised centuries-old encyclopaedias. In the early days of Web2, Flickr, Tumblr and DeviantArt were used by artists to share their work with a newly emerging distributed audience. The shift from Web1 to the Web2 platform economy model turned everyone into a ‘content creator’, whether they were artists or not, all whilst centralising bodies of data, which today serve as training data for AI models.6 In the context of large datasets such as Common Crawl, artists’ data, from fan art to install shots, are indistinguishable from other data in a repository, even though the conditions of production and the relationship to culture may be very different. However, there are a number of strategies emerging for artists specifically to meet the challenges of the ‘data as shadow labour’ regime.

Leverage IP for Opt In Returns

IP litigation from artists (predominantly in the USA) claims that AI training uses artists’ work without permission, and without the financial returns they are entitled to under law. In the USA, at least 10 cases were launched against AI developers and owners in 2023. One landmark case has been brought to the UK courts by Getty Images.7 Many cases are still in progress, but the process of litigation points to deeper issues of transparency and responsibility in a creative economy being transformed by AI. Nondisclosure of training data prevents individuals from knowing how their works are used. Litigation has the potential to create a route for entities that develop AI systems to become transparent about training data.8

However, the longer-term efficacy of litigation is questionable given high costs and resource barriers that most artists cannot overcome, as well as the potential of pending judgments favouring AI companies.

I think we can be quite imaginative in how we use the IP framework of recognition and value in a creative work, and have more innovative ways of compensating and remunerating people in that process.

- Reema Selhi

Leverage Spawning for Opt Out Bargaining Power

One alternative is to foster an environment for meaningful bargaining and collective action. IP has great utility in this case, but safeguards are needed to prevent monopolisation by platforms that constrain downward revenue streams to the point that remuneration to most artists becomes negligible. Such safeguards can’t come from quantifiable mechanisms like pricing, but instead need to emerge from such tried and tested routes as collective bargaining.

The brainchild of founder Jordan Meyer and artists Holly Herndon and Mat Dryhurst, Spawning introduces a consent layer to the internet by giving users the possibility to opt-out from their data being used in training.9 Spawning’s browser extension allows users to add any media to the Do Not Train registry and to highlight content that has already been registered. Complementary software can then block scrapers from accessing website contents, and from searching for content in popular data sets. Spawning targets internet users as well as developers. StabilityAI has integrated Spawning into their workflow to cross-check their training data sets before training models.10

While the opt-out strategy may resemble a protectionist measure that is only as good as the willingness of the actors behind the training to enforce it, it offers a new form of bargaining power not just to artists but to all internet users.

Data Brokerage for a Networked Commons

With an increasing awareness of data as shadow labour, data brokerage is likely to emerge as an AI-era form of collective bargaining. In sharp contrast to the individualising tendencies of the art market, artists now have an opportunity to collectivise around their economic value as training data producers and to test new modes of organising with trusted data stewards (see Chapter 1), such as forming data unions or data trusts.11 While such entities may need to be hosted and/or steered by existing organisations (see Leverage Spawning for Opt Out Bargaining Power above), if adopted widely across cultural communities, they could set an important precedent for wider public utility and common sense understanding of the role that such intermediating entities could play within an AI-driven economy.

Emerging Accountability Mechanisms

When the outputs generated from such tools as Midjourney, Stable Diffusion and ChatGPT are considered as artistic artefacts, questions of provenance, permission and attribution emerge. The IP of the artefact produced by generative AI is currently either automatically in the public domain (e.g. Midjourney), or is assigned to the prompter (e.g. GPT-4).12 Traceability, accountability and attribution norms will be critical technical and cultural frontiers as new mechanisms are introduced to support traceability and to combat permissionless use of training data, AI models, and their outputs (see Cultivating Trust and Verifiability in Chapter 1). Today these frontiers include the use of blockchain technologies to track machine learning provenance,13 metadata standards such as OpenAI C2PA metadata framework,14 and watermarking and identification tools such as SynthID by DeepMind.15 Where the traceability of data is opaque and unaccountable, techniques have been developed such as Kudurru developed by Spawning which poisons or attacks the network as a form of resistance to data malfeasance.16 17

The concept of ‘crafting’ as an ‘activity involving skill in making things by hand’ may seem anachronistic for the era of contemporary art - in which art-making is generally led by conceptual rather than skill-based concerns - and too anthropocentric for the era of AI.18 However, the historical space of art and (advanced) technologies has always differentiated itself from contemporary art precisely in its craft-like sensibility towards technological systems, whereas the merging of human and machine logics and capabilities highlights a synthesis that artists interested in advanced technologies have been forging since at least the 1960s.19 20 21

These tools offer access and insight into the various capabilities of deep learning models. For instance, ChatGPT is an interface into LLMs such as GPT-4 and their transformer architecture. Other tools that use API access to build on top of GPT-4 may be fine-tuned on additional expert information (e.g. CaseText, a legal advice chatbot), but they are indebted, both financially and technically, to the foundation model.22 Some tools even aggregate foundation models, e.g. Perplexity, a co-pilot interface which pulls from both GPT-4 and Claude. Through using - and jailbreaking - the tool, insight is gained into its affordances as well as its limits.23

Iteration at the Core of Synthetic Crafting

Iterative and durational learning are intrinsic to crafting for humans and for machine learning. Refinement follows reflection, and is never finished, it merely pauses. Every time a model is prompted, it ‘runs a course’ of the model architecture. Different prompts can produce variable results. These iterative runs can be seen as agents completing a task. With each prompt an agent produces, learnings that inform the next agent are produced. Working iteratively can lead to a more profound understanding of what the tools offer, as it allows for testing them out and taking them to new places.

This type of commitment to working closely with a technological system can give those engaged in synthetic crafting unique insights through competency, reciprocal learning, and higher exposure to the glitches and edge cases that a new model might reveal over time. It also means that artists who have skills that predate AI’s advance, and that speak to the sensibilities being harnessed, have an essential knowledge base for mastering and expanding the possibilities of these tools. For example, when Adobe Suite was first released, painters, photographers and physical retouchers were well placed to leverage and explore the tools as they had a deep understanding of how image production works.24

Edging into Unchartered Territory

Typically, we are inclined to downgrade models as we believe these older, less optimised versions, offer untapped potential for exploring unique sensibilities and narratives, a feature that the newer commercial models oftentimes don’t accommodate.

— CROSSLUCID

We are still at the very beginning of understanding the particular affordances of AI technologies. Though certain model architectures are developed for specific purposes, their long-tail affordances might emerge later. In such instances, the core affordances of a model are only discovered when it is adapted from one deployment context - often its original intended purpose - and placed in a new one.25 Artistic experimentation often plays the role of repurposing tools from one context to another, finding a new perspective on an existing operation, or a latent operation within a new system of meaning.

From Deskilling to Reskilling

Delegating skilled labour to a black box technology potentially reduces agency over production processes. What is found in efficiency gains risks exposure to deskilling, and could disincentivise a sustained engagement in honing a pre-existing craft or set of skills. Certain skills, such as animation or the writing of code, are becoming subject to varying degrees of automation, putting a wider range of present-day skill-sets and forms of labour at risk.

However, historically, the loss and acquisition of human skills has always been a dynamic process; while some skills may become less widespread, others proliferate, and yet others make a come-back following an era of near total disappearance as cultural actors seek novel tools and ways of working, and as social and market demands shift.26 What pre-AI crafts will be preserved and who will benefit from those efforts will be matters for the joint efforts of communities, government, cultural institutions, and private investment.

Though the impact of the new AI tools is still unfolding, a number of specific consequences can already be identified. Tendencies in modern and contemporary art led to macrotrends of ‘embodied’ deskilling, refocusing on cognitive or conceptual contributions in artistic practice. This yielded a loosening of constraints, important for both conceptual and new media practices in the 20th century, and the turn towards research-driven practice more recently. With this latter phase, and the internal pressures of the artistic labour market, came the need for maintaining the stereotypical ‘knowledge worker’ skill set, while simultaneously requiring ‘hard’ practical and technical skills.

The need for technical skills and digital literacy have led to a reskilling phase in education. For example in the computational arts, which are often self-taught, alongside the growing sector of specialised institutional educational programmes such as those provided at the School of Digital Arts, Manchester Metropolitan University, University of the Arts London, as well as New York University’s Tisch School of the Arts.27 This wider reskilling has not been limited strictly to computational arts, and, in many cases, lean on fluency in research methodologies from both the social sciences and the humanities, from ethnography to data analysis.28

Overall, this reskilling has culminated in a proliferation of specialisations within the machine learning domain. Here, specific expertise and literacies - e.g. in game AI, synthetic data, generative systems, language models, specific coding languages for integration work, such as C++ for machine learning and Unreal Engine - can be combined with specific artistic or cultural skills or knowledge to produce re-triangulated specialisations. Consider, for example, the case of the poet Sasha Stiles who works fine-tuning small language models, or the graphic designer Eric Hu who builds bespoke generative systems for the layout of assets.29 New mixes of divergent skills lead to new possibilities for building a unique micro-disciplinary space for exploration.

From Model Querying to Seamless Intermediation

The ascent of prompt-engineering as an AI-era craft-based skill is another example which requires a certain level of domain-specific knowledge/erudition.30 Prompt engineering has been compared to writing practice as a craft, but also as a form of conceptual labour.

Prompt-engineering can also be seen as an emerging method to ‘query’ an AI model as a search space - a contingent practice in drawing out important insights. Yet, prompt-engineering as it exists today might be specific to our moment, as newer more malleable interfaces are developed. Examples of potential futures might include user-friendly programming interfaces, such as the music app Semilla.ai, but also cloud service platforms such as NVIDIA’s Omniverse in which many models and software environments can be used together.31

The last decade has seen the re-emergence of artificial neural networks as the dominant computational paradigm in AI. Machine learning outputs from ‘early’ systems, for instance Google’s DeepDream, were initially interpreted as oneiric generative worlds, with their aesthetics capturing the imagination of the general public.32 With aesthetics acting as an interface to technical systems, something always remains encrypted, or autonomous from human intentionality.

Forays into aesthetic research, including data visualisations that surface and scrutinise the algorithmic tendencies of AI, became a core domain of artists’ work in the late 2010s.33 Here the ‘internal’ workings of the artificial neural network, and the dataset it pulls from, were the primary subjects of exploration. Semi-autonomous aesthetic outputs provided nonlinguistic insights into systems operating on the basis of non-symbolic pattern recognition.34 As a testament to the speed with which this technocultural space is evolving, this suite of tactics for visualising the operations of an artificial neural network has already been canonised even as new strategies (discussed below) emerge, and the anatomy of machine learning decision-making becomes increasingly theorised.35

Convergence Engines The stabilisation of a specific aesthetic can be

explained with recourse to the data that the model is trained on, but this does not tell the whole story. Machine learning architectures have an optimising logic which, in part, determines their outputs. This could be understood as the artificial neural network’s tendency to converge during the optimisation process. Optimisation strives to locate the ‘median’ result within the dataset. The delivery mechanism of algorithmic feeds produce a similar convergence effect: ‘mid’ content is self-perpetuating.36 In combination with other guardrails, some machine learning tools can become toothless for artistic purposes, imposing safeguarding constraints intended to delimit the possibility space for critically exploring especially challenging topics. Artists including Terence Broad are specifically interested in models that diverge, exploding new possibility spaces within the model’s typical logic.37

Style Capture Artists with a large enough presence in prevalent datasets

(e.g. Common Crawl or LAION-5B), or artists whose portfolios belong to a highly indexed dataset (in relation to a historical period, style, canon, etc.) might have their style ‘captured’ by generative AI models trained on relevant datasets. This can happen to an extent also when an artist or content producer works within a prevalent genre, and that genre is captured, with implications for the competitive viability of their work. This means that users can produce stylistic facsimiles by prompting models.38 This compression of artistic approach into a style transfer prompts the following questions: how will artists reconfigure their practice in the light of style capture? Will they use it to their advantage, intervening at the data layer to shape the model itself? Or will they evade it altogether by taking up strategies against data collection?

A framework for public claims on resources

Recombinant Aesthetics in the New Weird

It is not only extant styles that can be pulled from AI models. There is a possibility of composing new styles by exploring new connections between embeddings (i.e. compressed representations of data). This can be done in the vein of crossing styles, e.g. ‘Breaking Bad by Balenciaga,’ or it can allow for communities to play into their favourite tropes of fan lore while shifting their storylines in divergent directions (e.g. ‘Snow Crash but feminist’) creating forks, and adding fan-created storylines and characters.39 This poses the question as to whether there will be a need for new content (i.e. data points) to feed the variety of recombinant aesthetics, or whether recombinant aesthetics themselves will breed further emergence through mutations, or whether the interoperability of mutations will become what feeds the cultural desire for novelty.

The most important thing when we're talking with artists is to understand their workflow. They're creating new ways to use the technology in a pipeline. They're combining different algorithms, software and hardware and pushing the limits in a way we don’t often see.

- Heather Schoell

Different artists engage with AI tools in very different ways. Though there are many strategies for engaging or intervening at different levels of the AI stack, there is a continuum for proximity and/or depth of engagement, which leads to different forms of cross-pollination between artistic practices and the systems at the heart of evolving AI tools. The nature of engagement is not only determined by the artist’s technical skillset, but also by the nature of the tools themselves. Is the tool’s code open source? Is a pre-trained model freely available?40 Is the model’s training data auditable? Depending on the technical accessibility of a tool’s many sides, different forms of intervention at different layers of the stack might be made possible.

A framework for public claims on resources

The different modalities in which models are intervened upon as a medium by artists.

Modalities for AI Tool Deployment

There is already a long history of open machine learning tools that coder-artists have been using.41 In many such cases, these artists attempt to understand how a tool works, peering into its internal operations. Given the multiplicity of ways in which such artists interrogate AI tools, it is useful to break down a nomenclature for modes of engagement as practice modalities. Artists who use proprietary tools to produce artefacts are users, while those who embed those tools in more complex workflows are advanced users. If a tool is open source, some artists become auditors and are able to offer more technical scrutiny or feedback on how tools are built, either for the developer, a wider user community, or in some cases, a decentralised community of developers working on specific larger ventures.42

Again, if the code is open source a very small minority of artists, with the right support, can build out alternatives. An example of this is KaiberAI, a generative video tool that was developed through support from Gray Area, an art and technology organisation based in San Francisco.43 Like KaiberAI, though, not all competitors remain open source: some choose to bring their tool to market as proprietary and commercial. Conversely, there is SemillaAI by Moisés Horta Valenzuela, an audio synthesis tool that diversifies the landscape of generative AI tools for sonic practice by adding an additional tool available to musicians. The tool itself is open source, and allows any user, or artist, to train their own model based on their own audio repository. A user may choose this tool over a different generative music tool, thus having been provided with an alternative to proprietary, commercial products. In this sense, SemillaAI is competing on the market with such products, to the benefit of the artistic community of users.

Model-Making as Meaning-Making

Architectures are generic; weights are indebted to labour, and are much more idiosyncratic.

- Moisés Horta Valenzuela

In the age of foundation models, computations across unprecedentedly vast cultural archives derive meaning through correlations within a model’s embedding spaces. If culture and art-making are to play the role of diverging from inherited convention toward new modes of meaning-making, some of that work will need to take place within the technical system of meaning production itself, such is the AI model.44 Artists as cultural tastemakers and pathfinders will need to find ways to intervene in the meaning-making of AI systems by being able to manipulate the metadata, context, weights, and embeddings.45

The manipulation of AI models through weights - known as fine-tuning - holds unique potential for artists to perform interrogative work. The architecture of a neural network (i.e. GAN, CNN, transformer, etc.) only becomes a valuable meaning-making agent after training on data. As such, it is the training and fine-tuning of an AI model that engenders the formal aspects of artistic expression whether individual or collective. Working in this mode, artists become gatherers of datasets, explorers of embedding spaces, engineers of new architectures, and fine tuners of models. They use their own, existing and combinations of models in their production pipelines. The schematic ontologies as to where artistic expression originates, who is its rightful holder, and on what terms (e.g. IP and moral rights), will have major bearing on what we consider art to be, how it is to be legally and economically acknowledged, and, in turn, what the agency of its meaning-making in society could be.46

Systems Builders

Machine learning tools are being integrated into broader artistic complex systems which function beyond the production of visual or textual artefacts. The integration of AI and machine learning within simulations, video games, sensory apparatuses and surveillance systems mirrors the technology’s wider societal deployment, where AI becomes more embedded, rather than being the focal point of the artist’s attention.

- Creative AI Lab

Some artists, instead of looking inward at the internal operations of AI tools, look outward, building tools into more complex, operative systems for research or production. These artists craft tools, craft with tools, or craft tools into bigger systems. Some artists build interesting pipelines without notable technical knowledge by pulling together proprietary tools and techniques into novel workflows.47

This approach, which takes the system as an artform, is less common than methods which give primacy to some combination of process and artefact. An example is the work of 0rphan Drift and Etic Lab’s Interspecies Research Communication Initiative (2021) which aimed to build a reciprocal training programme for a machine learning agent and an octopus, adopting what they termed a ‘whole-systems approach’.48

From artistic practice emerges new approaches to technical and narrative aspects of machine learning, continuing a lineage of artistic endeavours of not merely critiquing but building (computational) systems as tools for communication and analysis. Often these systems become a byproduct that has an ecosystem level value, either by pressing forward with a specific technical innovation—for example, Delta_Ark’s work to integrate reinforcement learning into Unreal Engine - or the delivery of a tool for the community to build into future workflows, as with SemillaAI. From an organisational point of view, these projects often yield public interest and sometimes have positive impacts in their pursuit of divergent technical systems that decouple from the profit motive. However, they do require much greater resourcing, since they often necessitate teams of diverse technical and academic expertise.

We're trying to simulate ecosystems with agents that are highly adaptive. So that might mean agents with LLMs that you can talk to. But it also means that there are agents that can evolve new behaviours. We're trying to make these really complex simulated ecosystems, and trying to make artworks from inside them.

- Delta_Ark

Recombinant Bubble Worlds

Artists’ modus operandi cannot be generalised. They are always seeking new ways of working, and this often means combining technologies and bringing different technical components into interaction as part of new workflows. Increasingly, this involves building tools into worlds.49 Sometimes these worlds explore AI through narrative, such as Lawrence Lek’s Sinofuturist cinematic universe that privileges speculative fictional exploration over forensic exploration of tools. Alice Bucknell combines narrative with AI tooling which they use to constitute containerised gameworlds, integrating a composite stack of multiple models and techniques, for example, in their latest project The Alluvials (2024).

Game engines, like Unreal Engine or Unity, are currently the software environment through which experimental workflows and combinations of tooling are developed in order to build virtual worlds.50 Simulating worlds also allows for the testing of ideas beyond physical constraints e.g. limited resources or the laws of physics. AI will have a key role to play in the proliferation of virtual worlds as generative AI tools and recombinant media will present opportunities to create hyper-personalised versions of worlds, virtual environments and characters according to the specifications of creators or communities.51 52 53 With users becoming both producers and consumers of generative media we might see a proliferation of ‘bubble worlds’. These forked and user-specific versions of virtual worlds will be continually reshaped by their attendant communities or could join together to share in-world generative assets. 54

Agent-training Arenas

A lifeform, or a life, is a compositional space.

- Ian Cheng

Simulated bubble worlds provide a place to develop both new machine learning techniques and new philosophies for understanding them.55 Using simulation as an accelerator for emerging paradigms of weird social organisation. This type of work operates as a ‘mirror stage’ in which to explore emerging hybrid human-AI patterns of agency, prototyping new relational models which will eventually produce new, dialogical forms of personal and organisational self-recognition. This problem space has precedent in works of artists working in the 2010s, such as Ian Cheng, whose work BOB (2018) took the ‘life’ of an AI agent as medium itself.

This approach is not restricted to the cultural domain: simulations are used in other contexts to train AI agents. Startup Imbue (formally Generally Intelligent) developed their own simulated training environment called Avalon wherein they conduct benchmarking experiments for reinforcement learning agents.56

Complex System Choreography

This perspective on the artist - thinking across ‘system’ and ‘world’ - focuses more on the artist as performing a type of work with their unique skill set instead of viewing them as a metaphysically distinct type of agent who can offer opaque ethical insight into new future technologies. This echoes the art critic Jack Burnham’s 1968 essay System Esthetics: AI shifts the conventional role of the artist and creative away from end-content producer (primarily visual, imagery, text) towards a choreographer of complex information systems.57

We're not trying to compete, we simply intend to remain the hosting platform that keeps the community engaged and coming back.

- Madisen Taylor

The existence of art and technology as a professional nexus would have not been possible without a new set of structural relationships that were inaugurated in the 1950s involving the public sector, technology industry, and artists interested in technologies as an exploratory and conceptual medium.58 Experiments in Art and Technology at Bell Labs, Artist Placement Group, XEROX-Parc, Autodesk Artist In Residence and Google Artist + Machine Intelligence all serve as era-specific archetypes of public sector and industry-hosted settings where artists were invited to collaborate with technologists. While all of these programmes placed different expectations on such collaborations and often operated under different definitions of what an artist is and can offer, ranging from visual support to philosophical sparring partners, in aggregate they have set a precedent for the societal necessity of such relations that today’s AI public sector and industry landscape can leverage and evolve.

Exploratory Use-Case Space

The combination of artists’ appetite for risk-taking and experimentation with the low risks of art as a deployment context makes collaborations between technology developers and artists fertile soil for use-case exploration. This is true for the product-ready level of technology, but even more for R&D or community-building phases around early-stage technological applications. The specific affordances of artistic training and socialisation skills allow artists to approach new technologies with critical perspective, in some cases, including experience in cross-deploying new technologies in unexpected contexts. Under the right conditions, this can be used effectively, revealing new affordances by means of experimental use.

Apart from surfacing valuable insights and use-case prototyping, case studies can help develop a public-facing narrative around the technology. In the context of a highly abstract and complex technology such as AI, both of these capabilities are of great value and public significance. Key to the success of such programmes going forward will be the recognition of the inherent value provided by artists and arts organisations in these experimental contexts.

Artist as Influencer

Technologies oriented towards creative pursuits will inevitably leverage artists in their demonstration campaigns.59 In the generative art space, artists can endorse the use of particular models, particularly if their authorship is useful in creating value through tradable assets.60 More generally, the production of cultural artefacts (poems, fan fiction, imagery) has been the way that a general audience interfaces with foundational models and their logic systems.

Case for Institutional Brokerage and Intermediation

As generative as collaborations between artists and industry can be, they are fraught with potential misalignments. Contractual misunderstandings, cultural differences, power imbalances and the sheer asymmetry of operating scales between typical artist studios and technology companies often lead to frictions and even collaborative breakdowns. Furthermore, within the technology sector, such programmes are usually initiated and argued for through business cases by employees who see merit in such collaborations, but whose main job lies elsewhere. The relative fragility and lack of longevity of such programmes is, in part, a result of these operational challenges.

In contrast to the 1950s, 1990s, and even the 2010s, today there are a number of institutions, as well as smaller organisations, who specialise in supporting artistic practices that are specifically engaged with advanced technologies such as AI.61 These organisations possess the necessary capabilities to provide the operational scaffolding for such industry to artist collaborations as well as offering creative directional support and avenues for integrating collaborations into a larger public context.

Footnotes

  1. Creative-ai.org database and ml4a.net are two examples of resources for artists wanting to learn how to engage with AI tools.

  2. IMF, _Gen-AI: Artificial Intelligence and the Future of Work _(2024) [link].

  3. DACS, AI and Artists’ Work (2024), p.11 [link].

  4. “Whether a union exists or not, industry and labour can establish formal mechanisms to incorporate workers into AI development and deployment.” See Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (2023) [link].

  5. There are also clear positive externalities of such tools being integrated into artists’ administrative processes (which are typically significantly under-resourced). The streamlining and optimising of grant-writing, artist’s statements, bios, and other collateral communication materials associated with maintaining an artistic practice are examples.

  6. Early Web2 platforms such as DPChallenge were later adopted as training data. See Leonardo Impett, Touch Up My Appearance (2023).

  7. Emilia David, _Getty Lawsuit Against Stability AI to Go to Trial in the UK _(2023) [link].

  8. Michael M. Grynbaum and Ryan Mac, The Times Sues OpenAI and Microsoft Over A.I. Use of Copyrighted Work (2023) [link].

  9. See Spawning.ai [link.].

  10. Melissa Heikkilä, Artists Can Now Opt Out Of The Next Version Of Stable Diffusion (2022) [link].

  11. Sylvie Delacroix, Neil D Lawrence, Bottom-up Data Trusts: Disturbing The ‘One Size Fits All’ Approach To Data Governance, International Data Privacy Law, Volume 9, Issue 4, November 2019, p. 236-252, [link].

  12. OpenAI, _Terms of Use _(2024) [link].

  13. Muhammad Shafay et al., Blockchain For Deep Learning: Review And Open Challenges (2023) [link].

  14. OpenAI, C2PA in DALL·E 3, [link].

  15. DeepMind, SynthID, [link].

  16. Spawning, Kudurru, [link].

  17. See also Nightshade tool, [link].

  18. Peter Osborne, The Postconceptual Condition (2018).

  19. Maurice Tuchman, A Report on the Art and Technology Program of the Los Angeles County Museum of Art, 1967-1971 (1971).

  20. Sadie Plant, Zeroes and Ones (1997).

  21. Geoff Cox et al., Live Coding: A User's Manual (2022).

  22. i.e. fine-tuning on specific datasets which can offer a specialist level of insights into a specific remit of information such as a model that is fine-tuned on case law and a set of specific legal cases within specific jurisdictions.

  23. ‘Jailbreaking’ methods circumnavigate developer restrictions, usually at the software level, to broaden the range of possibilities a user can achieve with a tool.

  24. See Quantel Paintbox, a forerunner to Photoshop, early access to which was granted to Richard Hamilton. Lev Manovich,_ Software Takes Command_ (2013).

  25. For example AlphaGo, a machine learning software initially designed to play the board game Go has been redeveloped into the architecture used for AlphaFold, a tool able to accurately predict the 3D models of protein structures, a capability that presumed to be able to accelerate research in the development of novel medicines including for Alzheimer’s disease. See Demis Hassabis, AI Could Solve Some of Humanity’s Hardest Problems. It Already Has. (2013) [link].

  26. See Leroi-Gourhan, André, Gesture and Speech (1964 [1993]).

  27. The School of Digital Arts at MMU was a £35m investment into an art school with state of art production facilities focusing on media production studios, immersive technologies, a gallery and music spaces that opened in 2021.

  28. The breadth of intersecting methodologies with AI from, the humanities and beyond, can be explored via the ‘Resources’ strand of the Creative AI Lab Database [link].

  29. See Sasha Stiles's practice [link]. See Allison Parrish’s practice [link].

  30. Prompt engineering commonly refers to shaping the text prompt given to a foundation model in a particular way, in order to shape the results it returns. This can involve adding clarifications or additional information to steer it away from a default path already identified by users of that model.

  31. See Moisés Horta Valenzuela’s more malleable interface SemillaAI [link].

  32. For example, Gray Area and Google’s exhibition DeepDream: _The Art of Neural Networks _(2016).

  33. See Anna Riddler’s Traces of Things (2018) or Refik Anadol’s Archive Dreaming (2017).

  34. Where early AI systems used symbolic AI or rule-based AI, to define the terms of intelligent systems, connectionist AI allowed associations to emerge through a model’s processing of datasets. See Geoffrey E. Hinton, Preface to the Special Issue on Connectionist Symbol Processing (1990) [link].

  35. Scott Robbins, A Misdirected Principle with a Catch: Explicability for AI (2019).

  36. Günseli Yalcinkaya, _How did everything get so mid? _(2023) [link].

  37. Broad et al., Active Divergence with Generative Deep Learning -- A Survey and Taxonomy (2021) [link]. See Terrence Broad’s website for more insight to their practice [link].

  38. For example text-to-image models such as DALL-E 3 can reproduce works in the style of famous painters, e.g. Hieronymus Bosch, although it is designed to refuse prompts that explicitly request the style of living artists. While it might refuse to produce works in the style of David Hockney, for example, there remain questions around whether it could be prompt engineered into doing so.

  39. demonflyingfox, Breaking Bad by Balenciaga (2023) [link].

  40. Open weights refers to releasing only the pretrained parameters or weights of the artificial neural network model itself.

  41. See Machine Learning for Art repository [link] and Serpentine & King’s College London’s Creative AI Lab Database [link].

  42. Artists who engage in ‘auditing’ also usually engage in other modes, too. See, for example, Delta_Ark’s engagement with reinforcement learning for Unreal Engine [link].

  43. Kaiber.ai [link]

  44. Mercedes Bunz, ‘The calculation of meaning: on the misunderstanding of new artificial intelligence as culture’ (2019) [link].

  45. Mercedes Bunz & Eva Jäger, _Inquiring the Backends of Machine Learning Artworks: Making Meaning by Calculation _(2021).

  46. Holly+ a model and artwork created by Holly Herndon and Mat Dryhurst illustrates the complexity of artists producing work across the AI tech stack. In this work, the artists created a dataset from consenting participants, training a model together with engineer Jules Leplace, fine-tuning it on Herndon’s own voice. The work also explores recombinant IP with a strategy for shared ownership of assets created using the model [link].

  47. For more on this see Alasdair Milne, ‘Re-engineering a Concept of Collaboration for Machine Learning Artistic Practice’ (thesis, forthcoming, 2024).

  48. Interspecies Communication Research Initiative (ISCRI): A Cephalopod ↔ Machine Encounter (2021) [link].

  49. See What Models Make Worlds: Critical Imaginaries of AI exhibition at Ford Foundation Gallery (2023). Curated by Mashinka Firunts Hakopian & Meldia Yesayan [link].

  50. See _Future Art Ecosystems 2: Art x Metaverse _(2022) [link]

  51. Agnieszka Polska proposes that AI tools might lead to the proliferation of individual worlds, or ‘paracosms’, democratising access to siloed world-dwelling. These paracosms could remain individual or become collective, depending on how one builds out the infrastructure. Agnieszka Polska,_ Future Paracosms and Their Infrastructures _(2023) [link].

  52. Elsewhere, Autonomous Worlds, for example, use blockchain protocols to allow worlds to become decoupled from singular ownership, affording them the possibility of persisting indefinitely. These technologies offer the possibility. See the *Autonomous Worlds *reader (2023) [link].

  53. See Deepmind’s ‘Scalable Instructable Multiworld Agent’ in A generalist AI agent for 3D virtual environments (2024) [link].

  54. The term ‘bubble worlds’ emerged from a conversation between the authors and Vincent Brisebois who heads Omniverse at NVIDIA about the future of the media ecosystem in March 2024.

  55. Whether these truth-claims can be exported into our world remains a primary challenge for simulated scenarios and worldbuilding artistic research agendas alike.

  56. Generally Intelligent, Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds (2022) [link].

  57. Jack Burnham, Systems Esthetics (1968) [link].

  58. W. Patrick McCray, _Making Art Work: How Cold War Engineers and Artists Forged a New Creative Culture _(2020).

  59. '“Alex [Reben] is one of the first people we share our new models with,” said Natalie Summers, a spokeswoman for OpenAI'. Reben was OpenAI’s first artist-in-residence in 2023. Leslie Katz, _An Artist in Residence on A.I.’s Territory _(2023) [link].

  60. See community-based marketplaces for on-chain art such as fx(hash) [link] or Zien [link].

  61. Apart from Serpentine Arts Technologies, such organisations include Gray Area in San Francisco, HEK in Basel, Museum for Moving Image and NEW INC in New York, Buffalo AKG Art Museum in Buffalo, TRANSFER Gallery and Bitforms Gallery, amongst others.