Instructions to use amphora/fc-reasoning-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use amphora/fc-reasoning-8b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="amphora/fc-reasoning-8b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("amphora/fc-reasoning-8b") model = AutoModelForCausalLM.from_pretrained("amphora/fc-reasoning-8b") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use amphora/fc-reasoning-8b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "amphora/fc-reasoning-8b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "amphora/fc-reasoning-8b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/amphora/fc-reasoning-8b
- SGLang
How to use amphora/fc-reasoning-8b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "amphora/fc-reasoning-8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "amphora/fc-reasoning-8b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "amphora/fc-reasoning-8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "amphora/fc-reasoning-8b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use amphora/fc-reasoning-8b with Docker Model Runner:
docker model run hf.co/amphora/fc-reasoning-8b
| {# version=v3-llama3.1 #}{%- macro append_new_param_info(param_declaration, comment_info, examples_info, depth) -%} | |
| {%- set offset = "" -%} | |
| {%- if depth >= 1 -%} | |
| {%- set offset = " " * depth -%} | |
| {%- endif -%} | |
| {%- if comment_info != "<|NONE|>" -%} | |
| {{ "\n" + offset + comment_info }} | |
| {%- if examples_info | length > 0 -%} | |
| {# Append each example info #} | |
| {%- for example in examples_info -%} | |
| {{ "\n" + offset + "// " + example|string|replace("'", '"') }} | |
| {%- endfor -%} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {{ "\n" + offset + param_declaration }} | |
| {%- endmacro -%} | |
| {%- macro convert_data_type(param_type) -%} | |
| {%- if param_type == "integer" or param_type == "float" -%} | |
| {{ "number" }} | |
| {%- else -%} | |
| {{ param_type }} | |
| {%- endif -%} | |
| {%- endmacro -%} | |
| {%- macro get_param_type(param) -%} | |
| {%- set param_type = "any" -%} | |
| {%- if "type" in param -%} | |
| {%- set raw_param_type = param["type"] -%} | |
| {%- if raw_param_type is iterable and raw_param_type is not string -%} | |
| {%- set param_type = raw_param_type | join(" | ") -%} | |
| {%- else -%} | |
| {%- set param_type = raw_param_type -%} | |
| {%- endif -%} | |
| {{ convert_data_type(param_type) }} | |
| {%- elif "oneOf" in param -%} | |
| {%- set one_of_types = param["oneOf"]|selectattr("type", "defined")|list -%} | |
| {%- set one_of_types = one_of_types|map(attribute="type")|unique|list -%} | |
| {{ convert_data_type(one_of_types | join(" | ")) }} | |
| {%- endif -%} | |
| {%- endmacro -%} | |
| {%- macro get_format_param(param) -%} | |
| {%- if "format" in param -%} | |
| {{ param["format"] }} | |
| {%- elif "oneOf" in param -%} | |
| {%- set formats = [] -%} | |
| {%- for item in param["oneOf"] -%} | |
| {%- if "format" in item -%} | |
| {%- if item["format"] == param["oneOf"][-1]["format"] -%} | |
| {{ item["format"] }} | |
| {%- else -%} | |
| {{ item["format"] + " or "}} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- else -%} | |
| {{ "<|NONE|>" }} | |
| {%- endif -%} | |
| {%- endmacro -%} | |
| {%- macro get_param_info(param) -%} | |
| {%- set param_type = param.get("type", "any") -%} | |
| {%- set format_param = get_format_param(param) -%} | |
| {%- if "description" in param or "default" in param or format_param != "<|NONE|>" or param["maximum"] or param["minimum"] or param["maxLength"] or param["minLength"] -%} | |
| {{ "//" }} | |
| {%- if "description" in param -%} | |
| {%- set desc = param["description"] -%} | |
| {%- if not desc.endswith(".") -%} | |
| {%- set desc = desc + "." -%} | |
| {%- endif -%} | |
| {{ " " + desc }} | |
| {%- endif -%} | |
| {%- if "default" in param -%} | |
| {%- set default_value = param["default"] -%} | |
| {%- if param_type == "string" -%} | |
| {%- set default_value = '"' ~ default_value ~ '"' -%} | |
| {%- endif -%} | |
| {{ " Default=" ~ default_value ~ "." }} | |
| {%- endif -%} | |
| {%- set format_param = get_format_param(param) -%} | |
| {%- if format_param != "<|NONE|>" -%} | |
| {{ " Format=" ~ format_param }} | |
| {%- endif -%} | |
| {%- for field, field_name in [("maximum", "Maximum"), ("minimum", "Minimum"), ("maxLength", "Maximum length"), ("minLength", "Minimum length")] -%} | |
| {%- if field in param -%} | |
| {{ " " + field_name ~ "=" ~ param[field] }} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- else -%} | |
| {{ "<|NONE|>"}} | |
| {%- endif -%} | |
| {%- endmacro -%} | |
| {%- macro get_enum_option_str(enum_options) -%} | |
| {%- for v in enum_options -%} | |
| {%- if v is string -%} | |
| {{ '"' + v + '"' }} | |
| {%- else -%} | |
| {{ v }} | |
| {%- endif -%} | |
| {%- if enum_options|length > 0 and v != enum_options[-1] -%} | |
| {{ " | " }} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- endmacro -%} | |
| {%- macro get_array_typescript(param_name, param_dic, depth) -%} | |
| {%- set offset = '' -%} | |
| {%- if depth >= 1 -%} | |
| {%- set offset = " " * depth -%} | |
| {%- endif -%} | |
| {%- set items_info = param_dic.get('items', {}) -%} | |
| {%- if items_info|length == 0 -%} | |
| {%- if param_name -%} | |
| {{ "\n" + offset + param_name + ": []" }} | |
| {%- else -%} | |
| {{ "\n" + offset + "[]" }} | |
| {%- endif -%} | |
| {%- else -%} | |
| {%- set array_type = get_param_type(items_info) -%} | |
| {%- if array_type == 'object' -%} | |
| {%- if param_name -%} | |
| {{ "\n" + offset + param_name + ": {" }} | |
| {%- else -%} | |
| {{ "\n" + offset + "{" }} | |
| {%- endif -%} | |
| {{ get_parameter_typescript(items_info.get('properties', {}), items_info.get('required', []), depth + 1) -}} | |
| {{- "\n" + offset + "}[]" }} | |
| {%- elif array_type == 'array' -%} | |
| {%- set item_info = get_array_typescript(None, items_info, depth + 1) -%} | |
| {%- if not param_name -%} | |
| {{ "\n" + item_info + "[]" }} | |
| {%- else -%} | |
| {{ "\n" + offset + param_name + ": " + item_info|trim + "[]" }} | |
| {%- endif -%} | |
| {%- else -%} | |
| {%- if 'enum' in items_info -%} | |
| {%- set item_type = get_enum_option_str(items_info['enum']) -%} | |
| {%- if param_name is none -%} | |
| {{ "(" + item_type + ")[]"}} | |
| {%- else -%} | |
| {{ "\n" + offset + param_name + ": (" + item_type + ")[]" }} | |
| {%- endif -%} | |
| {%- else -%} | |
| {%- if param_name is none -%} | |
| {{ "\n" + array_type + "[]" }} | |
| {%- else -%} | |
| {{ "\n" + offset + param_name + ": " + array_type + "[]," }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- endmacro -%} | |
| {%- macro get_parameter_typescript(properties, required_params, depth=0) -%} | |
| {%- set res = "" -%} | |
| {%- for param_name, param in properties.items() -%} | |
| {%- if param is mapping -%} | |
| {%- set comment_info = get_param_info(param) -%} | |
| {# Param Examples #} | |
| {%- set examples_info = [] -%} | |
| {%- if "examples" in param -%} | |
| {%- set examples_info = ["Example " + param_name + ":"] -%} | |
| {%- set examples_info = examples_info + param["examples"] -%} | |
| {%- endif -%} | |
| {# Param Name declaration #} | |
| {%- set param_declaration = param_name -%} | |
| {%- if required_params is iterable and param_name not in required_params -%} | |
| {%- set param_declaration = param_declaration + "?" -%} | |
| {%- endif -%} | |
| {%- set param_type = get_param_type(param) -%} | |
| {# Handle indentation based on depth #} | |
| {%- set offset = "" -%} | |
| {%- if depth >= 1 -%} | |
| {%- set offset = " " * depth -%} | |
| {%- endif -%} | |
| {%- if param_type == "object" -%} | |
| {%- if comment_info != "<|NONE|>" -%} | |
| {{ "\n" + offset + comment_info }} | |
| {%- endif -%} | |
| {%- if examples_info|length > 0 -%} | |
| {%- for example in examples_info -%} | |
| {{ "\n" + offset + "// " + example|string|replace("'", '"') }} | |
| {%- endfor -%} | |
| {%- endif -%} | |
| {%- set param_declaration = param_declaration + ": {" -%} | |
| {{ "\n" + offset + param_declaration -}} | |
| {{- get_parameter_typescript(param.get("properties", {}), param.get("required", []), depth + 1) -}} | |
| {{- "\n" + offset + "}," }} | |
| {%- elif param_type == "array" -%} | |
| {%- set item_info = param.get("items", {}) -%} | |
| {%- if "type" not in item_info -%} | |
| {%- set param_declaration = param_declaration + ": []," -%} | |
| {{ append_new_param_info(param_declaration, comment_info, examples_info, depth) }} | |
| {%- else -%} | |
| {%- if comment_info != "<|NONE|>" -%} | |
| {{ "\n" + offset + comment_info }} | |
| {%- endif -%} | |
| {%- if examples_info|length > 0 -%} | |
| {%- for example in examples_info -%} | |
| {{ "\n" + offset + "// " + example|string|replace("'", '"') }} | |
| {%- endfor -%} | |
| {%- endif -%} | |
| {%- set array_declaration = get_array_typescript(param_declaration, param, depth) -%} | |
| {%- if not array_declaration.endswith(",") -%} | |
| {%- set array_declaration = array_declaration + "," -%} | |
| {%- endif -%} | |
| {{ array_declaration}} | |
| {%- endif -%} | |
| {%- else -%} | |
| {%- if "enum" in param -%} | |
| {%- set param_type = get_enum_option_str(param["enum"]) -%} | |
| {%- endif -%} | |
| {%- if "nullable" in param and param["nullable"] -%} | |
| {%- set param_type = param_type + " | null" -%} | |
| {%- endif -%} | |
| {%- set param_declaration = param_declaration + ": " + param_type + "," -%} | |
| {{ append_new_param_info(param_declaration, comment_info, examples_info, depth) }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- endmacro -%} | |
| {%- macro generate_schema_from_functions(functions, namespace='functions') -%} | |
| {{ "// Supported function definitions that should be called when necessary.\n" -}} | |
| {{- "namespace " + namespace + " {\n\n" -}} | |
| {%- for function in functions -%} | |
| {%- if function.get("function") -%} | |
| {%- set function = function.get("function") -%} | |
| {%- endif -%} | |
| {%- set function_name = function.get("name") -%} | |
| {%- if function_name -%} | |
| {%- set description = function.get('description', '') -%} | |
| {%- set parameters = function.get('parameters', {}) -%} | |
| {{- "// " + description + "\n" -}} | |
| {{- "type " + function_name -}} | |
| {%- if parameters and parameters.get("properties") -%} | |
| {{- " = (_: {" -}} | |
| {%- set required_params = parameters.get("required", []) -%} | |
| {{ get_parameter_typescript(parameters.get("properties"), required_params, 0) -}} | |
| {{- "\n}) => any;\n\n" }} | |
| {%- else -%} | |
| {{ " = () => any;\n\n" }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {{ "} // namespace " + namespace }} | |
| {%- endmacro -%} | |
| {%- if not tools is defined -%} | |
| {%- set tools = none -%} | |
| {%- endif -%} | |
| {%- set has_code_interpreter = tools | selectattr("type", "equalto", "code_interpreter") | list | length > 0 -%} | |
| {%- if has_code_interpreter -%} | |
| {%- set tools = tools | rejectattr("type", "equalto", "code_interpreter") | list -%} | |
| {%- endif -%} | |
| {#- System message + builtin tools #} | |
| {{- bos_token + "<|start_header_id|>system<|end_header_id|>\n\n" }} | |
| {%- if has_code_interpreter %} | |
| {{- "Environment: ipython\n\n" }} | |
| {%- else -%} | |
| {{ "\n"}} | |
| {%- endif %} | |
| {%- if tools %} | |
| {{- "\nYou have access to the following functions:\n\n" }} | |
| {%- for t in tools %} | |
| {%- if "type" in t -%} | |
| {{ "Use the function '" + t["function"]["name"] + "' to '" + t["function"]["description"] + "'\n" + t["function"] | tojson() }} | |
| {%- else -%} | |
| {{ "Use the function '" + t["name"] + "' to '" + t["description"] + "'\n" + t | tojson }} | |
| {%- endif -%} | |
| {{- "\n\n" }} | |
| {%- endfor %} | |
| {{- '\nThink very carefully before calling functions.\nIf a you choose to call a function ONLY reply in the following format:\n<{start_tag}={function_name}>{parameters}{end_tag}\nwhere\n\nstart_tag => `<function`\nparameters => a JSON dict with the function argument name as key and function argument value as value.\nend_tag => `</function>`\n\nHere is an example,\n<function=example_function_name>{"example_name": "example_value"}</function>\n\nReminder:\n- If looking for real time information use relevant functions before falling back to brave_search\n- Function calls MUST follow the specified format, start with <function= and end with </function>\n- Required parameters MUST be specified\n- Only call one function at a time\n- Put the entire function call reply on one line\n\n' -}} | |
| {%- endif %} | |
| {{- "<|eot_id|>" -}} | |
| {%- for message in messages -%} | |
| {%- if message['role'] == 'user' or message['role'] == 'system' -%} | |
| {{ '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' + message['content'] + '<|eot_id|>' }} | |
| {%- elif message['role'] == 'tool' -%} | |
| {{ '<|start_header_id|>ipython<|end_header_id|>\n\n' + message['content'] + '<|eot_id|>' }} | |
| {%- else -%} | |
| {%- if (message['content'] and message['content']|length > 0) or ('tool_calls' in message and message['tool_calls'] and message['tool_calls']|length > 0) -%} | |
| {{ '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'}} | |
| {%- endif -%} | |
| {%- if message['content'] and message['content']|length > 0 -%} | |
| {{ message['content'] }} | |
| {%- endif -%} | |
| {%- if 'tool_calls' in message and message['tool_calls'] and message['tool_calls']|length > 0 -%} | |
| {%- for tool_call in message['tool_calls'] -%} | |
| {%- if tool_call["function"]["name"] == "python" -%} | |
| {{ '<|python_tag|>' + tool_call['function']['arguments'] }} | |
| {%- else -%} | |
| {{ '<function=' + tool_call['function']['name'] + '>' + tool_call['function']['arguments'] + '</function>' }} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {{ '<|eom_id|>' }} | |
| {%- elif message['content'] and message['content']|length > 0 -%} | |
| {{ '<|eot_id|>' }} | |
| {%- endif -%} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- if add_generation_prompt -%} | |
| {{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }} | |
| {%- endif -%} | |